'Transport indicated EOF'

[sajith]

When running tox with RabbitMQ launched by tox-docker (on #225 branch, not on main yet), some tests report failure. I want to look at this separately, so filing an issue now.

FAILED sdx_controller/test/test_connection_controller.py::TestConnectionController::test_delete_connection - pika.exceptions.IncompatibleProtocolError: StreamLostError: ('Transport indicated EOF',)
FAILED sdx_controller/test/test_connection_controller.py::TestConnectionController::test_getconnection_by_id - pika.exceptions.IncompatibleProtocolError: StreamLostError: ('Transport indicated EOF',)

Here's the detailed log:

ERROR    pika.adapters.utils.io_services_utils:io_services_utils.py:796 Socket EOF; <socket.socket fd=23, family=10, type=1, proto=6, laddr=('::1', 54130, 0, 0)>
ERROR    pika.adapters.base_connection:base_connection.py:445 Transport indicated EOF.
ERROR    pika.adapters.base_connection:base_connection.py:428 connection_lost: StreamLostError: ('Transport indicated EOF',)
ERROR    pika.connection:connection.py:2048 Probably incompatible Protocol Versions
ERROR    pika.adapters.utils.connection_workflow:connection_workflow.py:291 AMQPConnector - reporting failure: AMQPConnectorAMQPHandshakeError: IncompatibleProtocolError: The protocol returned by the server is not supported: ("StreamLostError: ('Transport indicated EOF',)",)
ERROR    pika.adapters.utils.io_services_utils:io_services_utils.py:796 Socket EOF; <socket.socket fd=24, family=2, type=1, proto=6, laddr=('127.0.0.1', 53560)>
ERROR    pika.adapters.base_connection:base_connection.py:445 Transport indicated EOF.
ERROR    pika.adapters.base_connection:base_connection.py:428 connection_lost: StreamLostError: ('Transport indicated EOF',)
ERROR    pika.connection:connection.py:2048 Probably incompatible Protocol Versions
ERROR    pika.adapters.utils.connection_workflow:connection_workflow.py:291 AMQPConnector - reporting failure: AMQPConnectorAMQPHandshakeError: IncompatibleProtocolError: The protocol returned by the server is not supported: ("StreamLostError: ('Transport indicated EOF',)",)
ERROR    pika.adapters.utils.connection_workflow:connection_workflow.py:746 AMQP connection workflow failed: AMQPConnectionWorkflowFailed: 2 exceptions in all; last exception - AMQPConnectorAMQPHandshakeError: IncompatibleProtocolError: The protocol returned by the server is not supported: ("StreamLostError: ('Transport indicated EOF',)",); first exception - AMQPConnectorAMQPHandshakeError: IncompatibleProtocolError: The protocol returned by the server is not supported: ("StreamLostError: ('Transport indicated EOF',)",).
ERROR    pika.adapters.utils.connection_workflow:connection_workflow.py:723 AMQPConnectionWorkflow - reporting failure: AMQPConnectionWorkflowFailed: 2 exceptions in all; last exception - AMQPConnectorAMQPHandshakeError: IncompatibleProtocolError: The protocol returned by the server is not supported: ("StreamLostError: ('Transport indicated EOF',)",); first exception - AMQPConnectorAMQPHandshakeError: IncompatibleProtocolError: The protocol returned by the server is not supported: ("StreamLostError: ('Transport indicated EOF',)",)
ERROR    pika.adapters.blocking_connection:blocking_connection.py:450 Connection workflow failed: AMQPConnectionWorkflowFailed: 2 exceptions in all; last exception - AMQPConnectorAMQPHandshakeError: IncompatibleProtocolError: The protocol returned by the server is not supported: ("StreamLostError: ('Transport indicated EOF',)",); first exception - AMQPConnectorAMQPHandshakeError: IncompatibleProtocolError: The protocol returned by the server is not supported: ("StreamLostError: ('Transport indicated EOF',)",)
ERROR    pika.adapters.blocking_connection:blocking_connection.py:457 Error in _create_connection().
Traceback (most recent call last):
  File "/home/sajith/projects/atlanticwave-sdx/sdx-controller/.tox/py3/lib/python3.12/site-packages/pika/adapters/blocking_connection.py", line 451, in _create_connection
    raise self._reap_last_connection_workflow_error(error)
pika.exceptions.IncompatibleProtocolError: StreamLostError: ('Transport indicated EOF',)
_____________________________________________________ TestConnectionController.test_getconnection_by_id ______________________________________________________

self = <sdx_controller.test.test_connection_controller.TestConnectionController testMethod=test_getconnection_by_id>
result = <TestCaseFunction test_getconnection_by_id>

    def __call__(self, result=None):
        """
        Does the required setup, doing it here
        means you don't have to call super.setUp
        in subclasses.
        """
        try:
>           self._pre_setup()

.tox/py3/lib/python3.12/site-packages/flask_testing/utils.py:136: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
.tox/py3/lib/python3.12/site-packages/flask_testing/utils.py:149: in _pre_setup
    self.app = self.create_app()
sdx_controller/test/__init__.py:21: in create_app
    app = create_app(run_listener=True if os.getenv("MQ_HOST") else False)
sdx_controller/__init__.py:76: in create_app
    create_rpc_thread(app)
sdx_controller/__init__.py:24: in create_rpc_thread
    app.rpc_consumer = RpcConsumer(thread_queue, "", app.te_manager)
sdx_controller/messaging/rpc_queue_consumer.py:23: in __init__
    self.connection = pika.BlockingConnection(
.tox/py3/lib/python3.12/site-packages/pika/adapters/blocking_connection.py:360: in __init__
    self._impl = self._create_connection(parameters, _impl_class)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <BlockingConnection impl=None>, configs = (<ConnectionParameters host=localhost port=5672 virtual_host=/ ssl=False>,)
impl_class = <class 'pika.adapters.select_connection.SelectConnection'>

    def _create_connection(self, configs, impl_class):
        """Run connection workflow, blocking until it completes.
    
        :param None | pika.connection.Parameters | sequence configs: Connection
            parameters instance or non-empty sequence of them.
        :param None | SelectConnection impl_class: for tests/debugging only;
            implementation class;
    
        :rtype: impl_class
    
        :raises: exception on failure
        """
    
        if configs is None:
            configs = (pika.connection.Parameters(),)
    
        if isinstance(configs, pika.connection.Parameters):
            configs = (configs,)
    
        if not configs:
            raise ValueError('Expected a non-empty sequence of connection '
                             'parameters, but got {!r}.'.format(configs))
    
        # Connection workflow completion args
        #   `result` may be an instance of connection on success or exception on
        #   failure.
        on_cw_done_result = _CallbackResult(
            namedtuple('BlockingConnection_OnConnectionWorkflowDoneArgs',
                       'result'))
    
        impl_class = impl_class or select_connection.SelectConnection
    
        ioloop = select_connection.IOLoop()
    
        ioloop.activate_poller()
        try:
            impl_class.create_connection(
                configs,
                on_done=on_cw_done_result.set_value_once,
                custom_ioloop=ioloop)
    
            while not on_cw_done_result.ready:
                ioloop.poll()
                ioloop.process_timeouts()
    
            if isinstance(on_cw_done_result.value.result, BaseException):
                error = on_cw_done_result.value.result
                LOGGER.error('Connection workflow failed: %r', error)
>               raise self._reap_last_connection_workflow_error(error)
E               pika.exceptions.IncompatibleProtocolError: StreamLostError: ('Transport indicated EOF',)

.tox/py3/lib/python3.12/site-packages/pika/adapters/blocking_connection.py:451: IncompatibleProtocolError
.tox/py3/lib/python3.12/site-packages/pika/adapters/blocking_connection.py:451: IncompatibleProtocolError
--------------------------------------------------------------------- Captured log call ----------------------------------------------------------------------
DEBUG    sdx_controller.utils.db_utils:db_utils.py:26 Trying to load sdx-controller-test-db from DB
DEBUG    sdx_controller.utils.db_utils:db_utils.py:29 No existing sdx-controller-test-db from DB, creating table
DEBUG    sdx_controller.utils.db_utils:db_utils.py:31 DB sdx-controller-test-db initialized
DEBUG    sdx_controller.utils.db_utils:db_utils.py:35 DB sdx-controller-test-db initialized
ERROR    pika.adapters.utils.io_services_utils:io_services_utils.py:796 Socket EOF; <socket.socket fd=26, family=10, type=1, proto=6, laddr=('::1', 54132, 0, 0)>
ERROR    pika.adapters.base_connection:base_connection.py:445 Transport indicated EOF.
ERROR    pika.adapters.base_connection:base_connection.py:428 connection_lost: StreamLostError: ('Transport indicated EOF',)
ERROR    pika.connection:connection.py:2048 Probably incompatible Protocol Versions
ERROR    pika.adapters.utils.connection_workflow:connection_workflow.py:291 AMQPConnector - reporting failure: AMQPConnectorAMQPHandshakeError: IncompatibleProtocolError: The protocol returned by the server is not supported: ("StreamLostError: ('Transport indicated EOF',)",)
ERROR    pika.adapters.utils.io_services_utils:io_services_utils.py:796 Socket EOF; <socket.socket fd=27, family=2, type=1, proto=6, laddr=('127.0.0.1', 53564)>
ERROR    pika.adapters.base_connection:base_connection.py:445 Transport indicated EOF.
ERROR    pika.adapters.base_connection:base_connection.py:428 connection_lost: StreamLostError: ('Transport indicated EOF',)
ERROR    pika.connection:connection.py:2048 Probably incompatible Protocol Versions
ERROR    pika.adapters.utils.connection_workflow:connection_workflow.py:291 AMQPConnector - reporting failure: AMQPConnectorAMQPHandshakeError: IncompatibleProtocolError: The protocol returned by the server is not supported: ("StreamLostError: ('Transport indicated EOF',)",)
ERROR    pika.adapters.utils.connection_workflow:connection_workflow.py:746 AMQP connection workflow failed: AMQPConnectionWorkflowFailed: 2 exceptions in all; last exception - AMQPConnectorAMQPHandshakeError: IncompatibleProtocolError: The protocol returned by the server is not supported: ("StreamLostError: ('Transport indicated EOF',)",); first exception - AMQPConnectorAMQPHandshakeError: IncompatibleProtocolError: The protocol returned by the server is not supported: ("StreamLostError: ('Transport indicated EOF',)",).
ERROR    pika.adapters.utils.connection_workflow:connection_workflow.py:723 AMQPConnectionWorkflow - reporting failure: AMQPConnectionWorkflowFailed: 2 exceptions in all; last exception - AMQPConnectorAMQPHandshakeError: IncompatibleProtocolError: The protocol returned by the server is not supported: ("StreamLostError: ('Transport indicated EOF',)",); first exception - AMQPConnectorAMQPHandshakeError: IncompatibleProtocolError: The protocol returned by the server is not supported: ("StreamLostError: ('Transport indicated EOF',)",)
ERROR    pika.adapters.blocking_connection:blocking_connection.py:450 Connection workflow failed: AMQPConnectionWorkflowFailed: 2 exceptions in all; last exception - AMQPConnectorAMQPHandshakeError: IncompatibleProtocolError: The protocol returned by the server is not supported: ("StreamLostError: ('Transport indicated EOF',)",); first exception - AMQPConnectorAMQPHandshakeError: IncompatibleProtocolError: The protocol returned by the server is not supported: ("StreamLostError: ('Transport indicated EOF',)",)
ERROR    pika.adapters.blocking_connection:blocking_connection.py:457 Error in _create_connection().
Traceback (most recent call last):
  File "/home/sajith/projects/atlanticwave-sdx/sdx-controller/.tox/py3/lib/python3.12/site-packages/pika/adapters/blocking_connection.py", line 451, in _create_connection
    raise self._reap_last_connection_workflow_error(error)
pika.exceptions.IncompatibleProtocolError: StreamLostError: ('Transport indicated EOF',)

Possibly related to #243. It is likely that this "problem" has been around a while, and it is just that tox-docker surfaced it now?