Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions docs/by-example/client.rst
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ replication. In order for clients to make use of this property it is
recommended to specify all hosts of the cluster. This way if a server does not
respond, the request is automatically routed to the next server:

>>> invalid_host = 'http://not_responding_host:4200'
>>> invalid_host = 'http://127.0.0.1:4201'
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Haven't investigated why, but on my machine resolving a hostname that doesn't exist takes way longer than having it resolve to localhost on a port where hopefully nothing runs.

Potential follow up would be to generally restructure this. http.rst and client.rst have lots of overlap.

>>> connection = client.connect([invalid_host, crate_host])
>>> connection.close()

Expand All @@ -49,7 +49,7 @@ It's possible to define a default timeout value in seconds for all servers
using the optional parameter ``timeout``. In this case, it will serve as a
total timeout (connect and read):

>>> connection = client.connect([crate_host, invalid_host], timeout=5)
>>> connection = client.connect([crate_host, invalid_host], timeout=1)
>>> connection.close()

If you want to adjust the connect- vs. read-timeout values individually,
Expand All @@ -58,7 +58,7 @@ please use the ``urllib3.Timeout`` object like:
>>> import urllib3
>>> connection = client.connect(
... [crate_host, invalid_host],
... timeout=urllib3.Timeout(connect=5, read=None))
... timeout=urllib3.Timeout(connect=1, read=None))
>>> connection.close()

Authentication
Expand Down
20 changes: 9 additions & 11 deletions docs/by-example/http.rst
Original file line number Diff line number Diff line change
Expand Up @@ -34,23 +34,23 @@ If no ``server`` argument (or no argument at all) is passed, the default one

When using a list of servers, the servers are selected by round-robin:

>>> invalid_host = "invalid_host:9999"
>>> even_more_invalid_host = "even_more_invalid_host:9999"
>>> http_client = HttpClient([crate_host, invalid_host, even_more_invalid_host], timeout=0.3)
>>> invalid_host1 = "192.0.2.1:9999"
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

>>> invalid_host2 = "192.0.2.2:9999"
>>> http_client = HttpClient([crate_host, invalid_host1, invalid_host2], timeout=0.3)
>>> http_client._get_server()
'http://127.0.0.1:44209'

>>> http_client._get_server()
'http://invalid_host:9999'
'http://192.0.2.1:9999'

>>> http_client._get_server()
'http://even_more_invalid_host:9999'
'http://192.0.2.2:9999'

>>> http_client.close()

Servers with connection errors will be removed from the active server list:

>>> http_client = HttpClient([invalid_host, even_more_invalid_host, crate_host], timeout=0.3)
>>> http_client = HttpClient([invalid_host1, invalid_host2, crate_host], timeout=0.3)
>>> result = http_client.sql('select name from locations')
>>> http_client._active_servers
['http://127.0.0.1:44209']
Expand All @@ -64,19 +64,17 @@ sleep after the first request::
>>> import time; time.sleep(1)
>>> server = http_client._get_server()
>>> http_client._active_servers
['http://invalid_host:9999',
'http://even_more_invalid_host:9999',
'http://127.0.0.1:44209']
['http://127.0.0.1:44209', 'http://192.0.2.2:9999', 'http://192.0.2.1:9999']
>>> http_client.close()

If no active servers are available and the retry interval is not reached, just use the oldest
inactive one:

>>> http_client = HttpClient([invalid_host, even_more_invalid_host, crate_host], timeout=0.3)
>>> http_client = HttpClient([invalid_host1, invalid_host2, crate_host], timeout=0.3)
>>> result = http_client.sql('select name from locations')
>>> http_client._active_servers = []
>>> http_client._get_server()
'http://invalid_host:9999'
'http://192.0.2.1:9999'
>>> http_client.close()

SQL Statements
Expand Down
34 changes: 17 additions & 17 deletions docs/by-example/https.rst
Original file line number Diff line number Diff line change
Expand Up @@ -36,22 +36,22 @@ With certificate verification

When using a valid CA certificate, the connection will be successful:

>>> client = HttpClient([crate_host], ca_cert=cacert_valid)
>>> client = HttpClient([https_host], ca_cert=cacert_valid)
>>> client.server_infos(client._get_server())
('https://localhost:65534', 'test', '0.0.0')

When not providing a ``ca_cert`` file, the connection will fail:

>>> client = HttpClient([crate_host])
>>> client.server_infos(crate_host)
>>> client = HttpClient([https_host])
>>> client.server_infos(https_host)
Traceback (most recent call last):
...
crate.client.exceptions.ConnectionError: Server not available, ...certificate verify failed...

Also, when providing an invalid ``ca_cert``, an error is raised:

>>> client = HttpClient([crate_host], ca_cert=cacert_invalid)
>>> client.server_infos(crate_host)
>>> client = HttpClient([https_host], ca_cert=cacert_invalid)
>>> client.server_infos(https_host)
Traceback (most recent call last):
...
crate.client.exceptions.ConnectionError: Server not available, ...certificate verify failed...
Expand All @@ -63,15 +63,15 @@ Without certificate verification
When turning off certificate verification, calling the server will succeed,
even when not providing a valid CA certificate:

>>> client = HttpClient([crate_host], verify_ssl_cert=False)
>>> client.server_infos(crate_host)
>>> client = HttpClient([https_host], verify_ssl_cert=False)
>>> client.server_infos(https_host)
('https://localhost:65534', 'test', '0.0.0')

Without verification, calling the server will even work when using an invalid
``ca_cert``:

>>> client = HttpClient([crate_host], verify_ssl_cert=False, ca_cert=cacert_invalid)
>>> client.server_infos(crate_host)
>>> client = HttpClient([https_host], verify_ssl_cert=False, ca_cert=cacert_invalid)
>>> client.server_infos(https_host)
('https://localhost:65534', 'test', '0.0.0')


Expand All @@ -85,22 +85,22 @@ The ``HttpClient`` constructor takes two keyword arguments: ``cert_file`` and
``key_file``. Both should be strings pointing to the path of the client
certificate and key file:

>>> client = HttpClient([crate_host], ca_cert=cacert_valid, cert_file=clientcert_valid, key_file=clientcert_valid)
>>> client.server_infos(crate_host)
>>> client = HttpClient([https_host], ca_cert=cacert_valid, cert_file=clientcert_valid, key_file=clientcert_valid)
>>> client.server_infos(https_host)
('https://localhost:65534', 'test', '0.0.0')

When using an invalid client certificate, the connection will fail:

>>> client = HttpClient([crate_host], ca_cert=cacert_valid, cert_file=clientcert_invalid, key_file=clientcert_invalid)
>>> client.server_infos(crate_host)
>>> client = HttpClient([https_host], ca_cert=cacert_valid, cert_file=clientcert_invalid, key_file=clientcert_invalid)
>>> client.server_infos(https_host)
Traceback (most recent call last):
...
crate.client.exceptions.ConnectionError: Server not available, exception: HTTPSConnectionPool...

The connection will also fail when providing an invalid CA certificate:

>>> client = HttpClient([crate_host], ca_cert=cacert_invalid, cert_file=clientcert_valid, key_file=clientcert_valid)
>>> client.server_infos(crate_host)
>>> client = HttpClient([https_host], ca_cert=cacert_invalid, cert_file=clientcert_valid, key_file=clientcert_valid)
>>> client.server_infos(https_host)
Traceback (most recent call last):
...
crate.client.exceptions.ConnectionError: Server not available, exception: HTTPSConnectionPool...
Expand All @@ -113,8 +113,8 @@ urrlib3 v2 dropped support for TLS 1.0 and TLS 1.1 by default, see `Modern secur
HTTPS requires TLS 1.2+`_. If you need to re-enable it, use the ``ssl_relax_minimum_version`` flag,
which will configure ``kwargs["ssl_minimum_version"] = ssl.TLSVersion.MINIMUM_SUPPORTED``.

>>> client = HttpClient([crate_host], ssl_relax_minimum_version=True, verify_ssl_cert=False)
>>> client.server_infos(crate_host)
>>> client = HttpClient([https_host], ssl_relax_minimum_version=True, verify_ssl_cert=False)
>>> client.server_infos(https_host)
('https://localhost:65534', 'test', '0.0.0')


Expand Down
4 changes: 2 additions & 2 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,6 @@ dev = [
"pytest<10",
"pytz",
"ruff<0.15",
"stopit<1.2",
]
docs = [
"sphinx",
Expand Down Expand Up @@ -107,7 +106,8 @@ non_interactive = true


[tool.pytest.ini_options]
addopts = "-rA --verbosity=3"
addopts = "-rA --verbosity=3 --doctest-modules"
doctest_optionflags = "NORMALIZE_WHITESPACE ELLIPSIS"
minversion = "2.0"
testpaths = [
"tests",
Expand Down
7 changes: 5 additions & 2 deletions src/crate/testing/layer.py
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,7 @@
import tempfile
import threading
import time
from typing import Optional

import urllib3

Expand Down Expand Up @@ -242,7 +243,7 @@ def __init__(
else:
self.http_url = http_url_from_host_port(host, port)

self.process = None
self.process: Optional[subprocess.Popen] = None
self.verbose = verbose
self.env = env or {}
self.env.setdefault("CRATE_USE_IPV4", "true")
Expand Down Expand Up @@ -364,7 +365,9 @@ def stop(self):
if self.process:
self.process.terminate()
self.process.communicate(timeout=10)
self.process.stdout.close()
stdout = self.process.stdout
if stdout:
stdout.close()
self.process = None
self.monitor.stop()
self._clean()
Expand Down
Loading
Loading