Adding the option to delete persisted user data as well.

[1kastner]

Proposed change

At https://discourse.jupyter.org/t/a-cull-idle-user-service-that-deletes-pvs/4742/ recently it was discussed that in some settings the persisted data of a user should also eventually be removed. This could be integrated into this service. I am really not sure whether it should because it is spawner-specific or even configuration-specific how to delete the persisted user data.

Alternative options

We could say that these concerns should be addressed separately and a second service could be created. The chances are pretty high that there would be more code duplication though to identify user accounts that exceed a certain age and haven't been used for a while.

Who would use this feature?

This is reasonable for settings with temporary users, e.g. mybinder or weekend seminars. You are sure you want to delete their data after some point.

[manics]

Together with #4 it almost sounds like we want a jupyter-admin cron utility. Perhaps with plugins for each function?

[1kastner]

For me that sounds like a reasonable approach. General plugins could be maintained in separate repositories to keep this clean. I hope from such a plugin I could still somehow reach the JupyterHub configuration so that the plugin can look up the details, such as which user "owns" which docker volume that should be deleted.

Regarding #4, in some cases it could make sense to link the plugin with the data source of the authenticator since that is a place e.g. email addresses are also stored. But again this might be very configration-specific. An alternative would be to have a separate plugin configuration file. From what I have seen until now I guess it violates the design principles of one centralized JupyterHub configuration though? Here I have a lack of experience with the JuypterHub design philosophy.

[meeseeksmachine]

This issue has been mentioned on Jupyter Community Forum. There might be relevant details there:

https://discourse.jupyter.org/t/a-cull-idle-user-service-that-deletes-pvs/4742/10

[1kastner]

At https://jupyterhub.readthedocs.io/en/stable/reference/services.html I checked that a service can not access the loaded JupyterHub configuration. I see three options out there and I don't like any of them:

1. Re-load the JupyterHub configuration inside this service to get the information regarding user-specific persisted data
2. Maintain a separate cleaning scripts which leads to duplicated configuration (what is the user's prefix? ...)
3. Refactor JupyterHub so that they can access the configuration and remove the previously constructed isolation

Any ideas on this?

[manics]

The JupyterHub API lets you obtain state: object. Arbitrary internal state from this server's spawner. Only available on the hub's users list or get-user-by-name method, and only if a hub admin. None otherwise. for each of a user's servers:
https://jupyterhub.readthedocs.io/en/stable/_static/rest-api/index.html#/definitions/Server

There's also a discussion about making auth_state from the Authenticator accessible too:
jupyterhub/jupyterhub#1704

Do you think the combination of these would allow a service to request the necesary information?

[1kastner]

In my example configuration here the important part regarding the Docker volumes is listed as such:

notebook_dir = '/home/jovyan/'
c.DockerSpawner.notebook_dir = notebook_dir
c.DockerSpawner.volumes = { 'jupyterhub-user-{username}': notebook_dir }

The data stored in c.DockerSpawner.volumes would be sufficient for my purpose of deleting docker volumes now but I can't speak for k8s configurations. For that I would need a partner on that side.

[1kastner]

So let's see if I got you right here: You suggest the DockerSpawner tells via the server state which docker volume belongs to it (in my case jupyterhub-user-{username}) and this (already serliaized) information is added to the auth_state but is only visible if you are admin (which the JupyterHub service is).

Once the JupyterHub service has the information jupyterhub-user-user0, it can also run docker volume rm jupyterhub-user-user0 and everything is fine and solved! So yes, that is a viable solution.

[manics]

Something like that! But this is outside my knowledge of JupyterHub, @minrk will know better.

[rkdarst]

I do something similar to this (if I understand correctly) to set unique cull times: https://github.com/AaltoSciComp/jupyterhub-aalto/blob/8bb8c3f0d538641141c5024c272245f943747fd2/scripts/cull_idle_servers.py#L180 https://github.com/AaltoSciComp/jupyterhub-aalto/blob/8bb8c3f0d538641141c5024c272245f943747fd2/jupyterhub_config.py#L772 ... requires a bit of care but in principle not too hard, and I think the idea works quite well.

[1kastner]

@rkdarst thank you so much for sharing that!

[meeseeksmachine]

This issue has been mentioned on Jupyter Community Forum. There might be relevant details there:

https://discourse.jupyter.org/t/a-cull-idle-user-service-that-deletes-pvs/4742/13

[1kastner]

Actually the server state is alreay published, see this code - thanks to all participants of this conversation to arrive there! Since the generic spawner api does not prescribe any spawner-specific content and the docker spawner does only add a little information, this only needs some additional information regarding the name of the docker volumes as presented before (see top).

Due to this input and also due to the discussion in the forum, I would suggest that I first create my personal variation of the dockerspawner that shares the information I need (i.e. the docker volume name) and second create a copy of this service using a different name. That service would only remove docker volumes if they are expired for a long time (we might want different times for culling idle Jupyter Notebooks and deleting whatever data the user believed we would persist for them).

For the long term, a plugin architecture as mentioned by manics sounds great to me too!