Error SSL: WRONG_VERSION_NUMBER when running scraper

[jerdog]

I am attempting to scrape my docusaurus website, and have setup the Typesense server on Docker locally on my Synology to test before deploying. I have it successfully reverse proxying https://sample-server.domain to the Typesense Docker instance.

Log:

curl https://sample-server.domain/health
{"ok":true}%

Here is my .env file for the scraper:

TYPESENSE_API_KEY=<api_key>
TYPESENSE_HOST=sample-server.domain
TYPESENSE_PORT=80
TYPESENSE_PROTOCOL=https

When I run the docker command to run the scraper against my website and dump to the server:

sudo docker run -it --env-file=.env -e "CONFIG=$(cat config.json | jq -r tostring)" typesense/docsearch-scraper:0.11.0

I get the following errors:

Traceback (most recent call last):
  File "/home/seleuser/.local/share/virtualenvs/seleuser-AdYDHarm/lib/python3.11/site-packages/urllib3/connectionpool.py", line 466, in _make_request
    self._validate_conn(conn)
  File "/home/seleuser/.local/share/virtualenvs/seleuser-AdYDHarm/lib/python3.11/site-packages/urllib3/connectionpool.py", line 1095, in _validate_conn
    conn.connect()
  File "/home/seleuser/.local/share/virtualenvs/seleuser-AdYDHarm/lib/python3.11/site-packages/urllib3/connection.py", line 730, in connect
    sock_and_verified = _ssl_wrap_socket_and_match_hostname(
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/seleuser/.local/share/virtualenvs/seleuser-AdYDHarm/lib/python3.11/site-packages/urllib3/connection.py", line 909, in _ssl_wrap_socket_and_match_hostname
    ssl_sock = ssl_wrap_socket(
               ^^^^^^^^^^^^^^^^
  File "/home/seleuser/.local/share/virtualenvs/seleuser-AdYDHarm/lib/python3.11/site-packages/urllib3/util/ssl_.py", line 469, in ssl_wrap_socket
    ssl_sock = _ssl_wrap_socket_impl(sock, context, tls_in_tls, server_hostname)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/seleuser/.local/share/virtualenvs/seleuser-AdYDHarm/lib/python3.11/site-packages/urllib3/util/ssl_.py", line 513, in _ssl_wrap_socket_impl
    return ssl_context.wrap_socket(sock, server_hostname=server_hostname)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/ssl.py", line 517, in wrap_socket
    return self.sslsocket_class._create(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.11/ssl.py", line 1108, in _create
    self.do_handshake()
  File "/usr/lib/python3.11/ssl.py", line 1379, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLError: [SSL: WRONG_VERSION_NUMBER] wrong version number (_ssl.c:992)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/seleuser/.local/share/virtualenvs/seleuser-AdYDHarm/lib/python3.11/site-packages/urllib3/connectionpool.py", line 789, in urlopen
    response = self._make_request(
               ^^^^^^^^^^^^^^^^^^^
  File "/home/seleuser/.local/share/virtualenvs/seleuser-AdYDHarm/lib/python3.11/site-packages/urllib3/connectionpool.py", line 490, in _make_request
    raise new_e
urllib3.exceptions.SSLError: [SSL: WRONG_VERSION_NUMBER] wrong version number (_ssl.c:992)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/seleuser/.local/share/virtualenvs/seleuser-AdYDHarm/lib/python3.11/site-packages/requests/adapters.py", line 667, in send
    resp = conn.urlopen(
           ^^^^^^^^^^^^^
  File "/home/seleuser/.local/share/virtualenvs/seleuser-AdYDHarm/lib/python3.11/site-packages/urllib3/connectionpool.py", line 843, in urlopen
    retries = retries.increment(
              ^^^^^^^^^^^^^^^^^^
  File "/home/seleuser/.local/share/virtualenvs/seleuser-AdYDHarm/lib/python3.11/site-packages/urllib3/util/retry.py", line 519, in increment
    raise MaxRetryError(_pool, url, reason) from reason  # type: ignore[arg-type]
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='sample-server.domain', port=80): Max retries exceeded with url: /collections/company-domain_1745776675 (Caused by SSLError(SSLError(1, '[SSL: WRONG_VERSION_NUMBER] wrong version number (_ssl.c:992)')))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/home/seleuser/src/index.py", line 138, in <module>
    run_config(environ['CONFIG'])
  File "/home/seleuser/src/index.py", line 45, in run_config
    typesense_helper.create_tmp_collection()
  File "/home/seleuser/src/typesense_helper.py", line 38, in create_tmp_collection
    self.typesense_client.collections[self.collection_name_tmp].delete()
  File "/home/seleuser/.local/share/virtualenvs/seleuser-AdYDHarm/lib/python3.11/site-packages/typesense/collection.py", line 22, in delete
    return self.api_call.delete(self._endpoint_path())
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/seleuser/.local/share/virtualenvs/seleuser-AdYDHarm/lib/python3.11/site-packages/typesense/api_call.py", line 158, in delete
    return self.make_request(requests.delete, endpoint, True,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/seleuser/.local/share/virtualenvs/seleuser-AdYDHarm/lib/python3.11/site-packages/typesense/api_call.py", line 129, in make_request
    raise last_exception
  File "/home/seleuser/.local/share/virtualenvs/seleuser-AdYDHarm/lib/python3.11/site-packages/typesense/api_call.py", line 103, in make_request
    r = fn(url, headers={ApiCall.API_KEY_HEADER_NAME: self.config.api_key}, **kwargs)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/seleuser/.local/share/virtualenvs/seleuser-AdYDHarm/lib/python3.11/site-packages/requests/api.py", line 157, in delete
    return request("delete", url, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/seleuser/.local/share/virtualenvs/seleuser-AdYDHarm/lib/python3.11/site-packages/requests/api.py", line 59, in request
    return session.request(method=method, url=url, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/seleuser/.local/share/virtualenvs/seleuser-AdYDHarm/lib/python3.11/site-packages/requests/sessions.py", line 589, in request
    resp = self.send(prep, **send_kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/seleuser/.local/share/virtualenvs/seleuser-AdYDHarm/lib/python3.11/site-packages/requests/sessions.py", line 703, in send
    r = adapter.send(request, **kwargs)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/seleuser/.local/share/virtualenvs/seleuser-AdYDHarm/lib/python3.11/site-packages/requests/adapters.py", line 698, in send
    raise SSLError(e, request=request)
requests.exceptions.SSLError: HTTPSConnectionPool(host='sample-server.domain', port=80): Max retries exceeded with url: /collections/company-domain_1745776675 (Caused by SSLError(SSLError(1, '[SSL: WRONG_VERSION_NUMBER] wrong version number (_ssl.c:992)')))

If I curl the docker instance directing without SSL it works:

curl http://localhost:8108/health
{"ok":true}%

Why is the application spitting the requests.exceptions.SSLError: HTTPSConnectionPool(host='sample-server.domain', port=80): Max retries exceeded with url: /collections/company-domain_1745776675 (Caused by SSLError(SSLError(1, '[SSL: WRONG_VERSION_NUMBER] wrong version number (_ssl.c:992)'))) error?

[jasonbosco]

This typically happens when you send http traffic to an https port or vice versa.

Could you try this change in your .env file:

TYPESENSE_API_KEY=<api_key>
TYPESENSE_HOST=sample-server.domain
- TYPESENSE_PORT=80
+ TYPESENSE_PORT=443
TYPESENSE_PROTOCOL=https