traefik started wrong when toml_static_config_file set

[Frank-Steiner]

Hi,

I tried to setup jupyterhub with traefik following https://jupyterhub-traefik-proxy.readthedocs.io/en/latest/toml.html
But it fails when using a different location for traefik.poml.

Bug description

When defining
c.TraefikTomlProxy.toml_static_config_file="/tmp/traefik.toml"
in jupyterhub_config.py and then starting jupyterhub, it says

[I 2023-01-11 16:55:33.326 JupyterHub toml:83] Starting traefik...
[E 2023-01-11 16:55:33.330 JupyterHub proxy:178] Error checking for traefik static configuration
    Traceback (most recent call last):
      File "/home/proj/software/jupyter_hub/lib/python3.10/site-packages/jupyterhub_traefik_proxy/proxy.py", line 176, in _check_traefik_static_conf_ready
        resp = await self._traefik_api_request("/api/providers/" + provider)
      File "/home/proj/software/jupyter_hub/lib/python3.10/site-packages/jupyterhub_traefik_proxy/proxy.py", line 160, in _traefik_api_request
        resp = await AsyncHTTPClient().fetch(
    tornado.httpclient.HTTPClientError: HTTP 401: Unauthorized

and repeats several times and finally dies with

[C 2023-01-11 17:03:54.197 JupyterHub app:3091] Failed to start proxy
    Traceback (most recent call last):
      File "/home/proj/software/jupyter_hub/lib/python3.10/site-packages/jupyterhub/app.py", line 3089, in start
        await self.proxy.start()
      File "/home/proj/software/jupyter_hub/lib/python3.10/site-packages/jupyterhub_traefik_proxy/toml.py", line 143, in start
        await self._wait_for_static_config(provider="file")
      File "/home/proj/software/jupyter_hub/lib/python3.10/site-packages/jupyterhub_traefik_proxy/proxy.py", line 190, in _wait_for_static_config
        await exponential_backoff(
      File "/home/proj/software/jupyter_hub/lib/python3.10/site-packages/jupyterhub/utils.py", line 236, in exponential_backoff
        raise asyncio.TimeoutError(fail_message)
    asyncio.exceptions.TimeoutError: Traefik static configuration not available

traefik.poml is correctly created in /tmp. Checking the process list there is one traefik process:
9772 pts/1 Z+ 0:00 [traefik] <defunct>
during this repeated error messages. It is gone after jupyterhub gives up.

When I manually start "traefik -c /tmp/traefik.poml" during the error messages appear, jupyterhub then succeeds:

[I 2023-01-11 17:06:14.588 JupyterHub app:3130] Hub API listening on http://127.0.0.1:8081/hub/
[I 2023-01-11 17:06:14.588 JupyterHub proxy:477] Adding route for Hub: / => http://127.0.0.1:8081
[I 2023-01-11 17:06:14.588 JupyterHub proxy:135] Waiting for / to register with traefik
[I 2023-01-11 17:06:14.647 JupyterHub app:3197] JupyterHub is now running at http://:8000

Thus, traefik as started by jupyterhub is somehow started in a wrong way so that it is not running correctly and jupyterhub cannot connect to it.

When I remove c.TraefikTomlProxy.toml_static_config_file option, the startup works fine and I see that a process
13108 pts/1 Sl+ 0:00 traefik -c /home/proj/software/jupyter_hub/lib/python3.10/site-packages/jupyterhub_traefik_proxy/traefik.toml
is running. This is somewhat strange, because the working dir I start jupyterhub in is /home/proj/software/jupyter_hub and that's where traefik.poml is indeed created. The file shown in the process list does not exist.

Your personal set up

• OS: SLES 15 SP4

• Version(s):
  bin/jupyterhub --version
  3.1.0

traefik/traefik version
Version: v1.7.28
Codename: maroilles
Go version: go1.14.13
Built: 2021-01-13_06:05:01PM
OS/Arch: linux/amd64

python3.10 --version
Python 3.10.8

jupyterhub_config.py is only:
c.Spawner.cmd = ['jupyter-labhub']
c.Spawner.default_url = '/lab'
c.JupyterHub.proxy_class = "traefik_toml"

cu,
Frank

[welcome]

Thank you for opening your first issue in this project! Engagement like this is essential for open source projects! 🤗

If you haven't done so already, check out Jupyter's Code of Conduct. Also, please try to follow the issue template as it helps other other community members to contribute more effectively.

You can meet the other Jovyans by joining our Discourse forum. There is also an intro thread there where you can stop by and say Hi! 👋

Welcome to the Jupyter community! 🎉