Add per-user audit logging for S3 CRUD actions

- New flask_s3_viewer.audit module emits one line per list/download/
  upload/delete/presign call with action, namespace, key, user, result,
  status_code, client_ip, user_agent, and a length-capped error field.
  Denied operations land at WARNING, exceptions at ERROR, success at
  INFO; control-byte sanitisation blocks log-line injection. emit() is
  a documented public helper for host integrations.
- Wire emit() into the blueprint: _enforce_auth fires before the
  401/403 abort, the @require decorator covers download/delete, and
  files/files_presign emit via try/finally. A request-scoped sentinel
  prevents duplicate or missing records, and the Google OAuth redirect
  path is intentionally silent (covered by a regression test).
- Move every direct logging.* call to module loggers and fix the
  long-standing PURGE / MKDIR / UP_OBJECT debug statements that were
  silently dropped because the object name was passed as a positional
  arg against an unparameterised message.
- Document the logger name, fields, level mapping, ProxyFix recommend-
  ation, RedactFilter / KeyErrorRedactFilter examples, and host-side
  emit() usage. Bump CHANGELOG with the next-release section.

Author: hidekuma

CHANGELOG.md

@@ -4,6 +4,59 @@ All notable changes to this project will be documented in this file. Dates are d
 
 Generated by [`auto-changelog`](https://github.com/CookPete/auto-changelog).
 
+#### [1.1.0] - unreleased
+
+> Audit logging & logging surface clean-up. The public API (`FlaskS3Viewer`
+> constructor, `add_new_one`, `init_app`, `get_instance`,
+> `get_boto_client`, `get_boto_session`) is unchanged — the changes
+> below affect log output only. The version line is provisional;
+> the final release tag is decided at release time.
+
+**Added**
+
+- New audit logger `flask_s3_viewer.audit`. Every S3 CRUD action that
+  flows through the blueprint (`list` / `download` / `upload` /
+  `delete` / `presign`) emits exactly one structured record carrying
+  `action`, `namespace`, `key`, `user`, `result`, `status_code`,
+  `client_ip`, `user_agent`, and (on failure) `error`. Level mapping:
+  `ok` → INFO, `denied` → WARNING, `error` → ERROR. Activation is
+  pure-`logging`: attach a handler to `flask_s3_viewer.audit`, set its
+  level, optionally pin `propagate = False`. No constructor flag.
+  `flask_s3_viewer.audit.emit(...)` is a public helper host code may
+  call to record extra non-CRUD operations on the same logger. Log
+  injection is defended at this layer — ASCII control bytes in any
+  user-controllable field (key / email / User-Agent / exception text)
+  are escaped to `\xNN` before the record is built. UA is capped at
+  256 bytes; free-form fields at 1024 bytes. See *Audit logging* in
+  `docs/source/usage/configuration.rst` for ProxyFix guidance, JSON
+  handler setup, and PII / ARN / bucket-name redaction filter
+  examples.
+
+**Changed**
+
+- Internal `logging.info(...)` / `logging.error(...)` /
+  `logging.debug(...)` calls in `flask_s3_viewer/__init__.py`,
+  `aws/s3.py`, `aws/session.py`, and `aws/cache.py` have been routed
+  through per-module `logging.getLogger(__name__)` instances. Record
+  `name` now reads `flask_s3_viewer.<module>` instead of `root`. Host
+  pipelines that filter by record `name` (rather than by handler
+  attachment) need to update their allow-lists; host pipelines that
+  attach handlers to the root logger (or to `flask_s3_viewer`) are
+  unaffected — the module loggers default to `propagate=True`.
+- `app.url_map` registration dump was demoted from INFO to DEBUG so
+  registering many namespaces (or large blueprints) no longer floods
+  INFO logs at app startup.
+
+**Fixed**
+
+- `aws/s3.py`'s `PURGE:` / `MKDIR:` / `UP_OBJECT:` DEBUG lines used
+  the broken `logging.debug('PURGE:', name)` form (the second
+  positional argument was treated as a logging-args tuple, against an
+  unparameterised message string, which the stdlib silently drops).
+  They now use `logger.debug('PURGE: %s', name)` etc., so a host that
+  pins the logger at DEBUG actually sees the keys involved in each
+  cache invalidation / placeholder create / upload event.
+
 #### [1.0.1](https://github.com/hidekuma/flask-s3-viewer/compare/1.0.0...1.0.1)
 
 > 15 May 2026

docs/doctrees/changelog.doctree

@@ -417,6 +417,191 @@ allow-list case — internally they wire up the
 ``permission_callback``. Pass your own ``permission_callback`` for
 fine-grained per-action policy.
 
+Audit logging
+-------------
+
+Every S3 CRUD action that flows through the blueprint emits a single
+structured record on the ``flask_s3_viewer.audit`` logger. The logger
+is always present — host applications opt in by attaching a handler
+and/or adjusting its level via the standard ``logging`` API. No
+constructor flag toggles audit on or off; the v1.0 public API is
+unchanged.
+
+**Logger name:** ``flask_s3_viewer.audit``
+**Default level:** unset (records propagate to root and are filtered
+by the host's effective level). Successful actions emit at
+``INFO``; permission denials emit at ``WARNING``; unexpected
+exceptions emit at ``ERROR``.
+
+**Record fields** (attached as ``LogRecord`` attributes via ``extra=``):
+
+  - ``action`` — one of ``list``, ``download``, ``upload``, ``delete``,
+    ``presign``
+  - ``namespace`` — viewer namespace the request landed on
+  - ``key`` — canonical S3 key / prefix (post-``base_path``)
+  - ``user`` — authenticated email or the literal string ``anonymous``
+  - ``result`` — ``ok`` / ``denied`` / ``error``
+  - ``status_code`` — HTTP status emitted to the client
+  - ``client_ip`` — ``request.remote_addr``
+  - ``user_agent`` — capped at 256 bytes; sanitised
+  - ``error`` — present only when an exception was attached
+
+The human-readable message is a single space-separated key=value line:
+
+.. code-block:: text
+
+    action=download namespace=fsv-test key=docs/report.pdf
+    user=alice@example.com result=ok status=200
+
+Newlines, carriage returns, and other ASCII control bytes inside
+attacker-controllable fields (key, email, User-Agent, exception
+message) are escaped as ``\\xNN`` before the record is built, so a
+crafted request cannot smuggle a fake row into the log stream.
+
+**Plain file handler example:**
+
+.. code-block:: python
+    :linenos:
+
+    import logging
+
+    handler = logging.FileHandler('/var/log/flask_s3_viewer/audit.log')
+    handler.setFormatter(logging.Formatter('%(asctime)s %(levelname)s %(message)s'))
+    logging.getLogger('flask_s3_viewer.audit').addHandler(handler)
+    logging.getLogger('flask_s3_viewer.audit').setLevel(logging.INFO)
+
+**Structured JSON handler example** (uses
+`python-json-logger <https://github.com/madzak/python-json-logger>`_):
+
+.. code-block:: python
+    :linenos:
+
+    import logging
+    from pythonjsonlogger import jsonlogger
+
+    audit_handler = logging.FileHandler('/var/log/flask_s3_viewer/audit.jsonl')
+    audit_handler.setFormatter(jsonlogger.JsonFormatter(
+        '%(asctime)s %(levelname)s %(action)s %(namespace)s '
+        '%(key)s %(user)s %(result)s %(status_code)s '
+        '%(client_ip)s %(user_agent)s'
+    ))
+    audit = logging.getLogger('flask_s3_viewer.audit')
+    audit.addHandler(audit_handler)
+    audit.setLevel(logging.INFO)
+    # The library leaves propagate=True by default — disable it here
+    # if you do NOT also want these records flowing to root handlers.
+    audit.propagate = False
+
+**PII / secret redaction.** Emails and S3 keys are written verbatim,
+which may be sensitive depending on deployment policy. Attach a
+``logging.Filter`` if you need to mask, hash, or drop fields before
+they hit disk — for example to GDPR-truncate the user field, or to
+strip ARNs/bucket names from ``error`` messages produced by boto3
+``ClientError`` stringification.
+
+.. code-block:: python
+    :linenos:
+
+    class RedactFilter(logging.Filter):
+        def filter(self, record):
+            if getattr(record, 'user', None):
+                user = record.user
+                record.user = user.split('@', 1)[0][:2] + '***@' + user.split('@', 1)[-1]
+            return True
+
+    audit.addFilter(RedactFilter())
+
+For ``key`` and ``error`` — which can carry full S3 paths and boto3
+``ClientError`` text containing bucket names / ARNs / request IDs —
+attach a second filter that keeps just enough breadcrumb to trace the
+incident without leaking the rest of the path or the AWS account
+topology:
+
+.. code-block:: python
+    :linenos:
+
+    import re
+
+    _ARN_RE = re.compile(r'arn:aws:[^\s"\']+')
+    _BUCKET_RE = re.compile(r'(?i)\bbucket[\s:=]+[^\s"\',]+')
+
+    class KeyErrorRedactFilter(logging.Filter):
+        """Redact prefix tails on ``key`` and AWS identifiers on ``error``."""
+        def filter(self, record):
+            key = getattr(record, 'key', None)
+            if key:
+                # Keep only the first path segment ("docs/...") so the
+                # audit trail still distinguishes top-level folders but
+                # the leaf filename / nested path is masked.
+                head, sep, _tail = key.partition('/')
+                record.key = f'{head}{sep}***' if sep else '***'
+            err = getattr(record, 'error', None)
+            if err:
+                err = _ARN_RE.sub('arn:aws:***', err)
+                err = _BUCKET_RE.sub('bucket=***', err)
+                record.error = err
+            return True
+
+    audit.addFilter(KeyErrorRedactFilter())
+
+The two filters compose — install both if you want the user, key, and
+error fields all masked. Tune the regex set to your environment;
+``ClientError`` text varies by API call.
+
+**Capturing the real client IP behind a reverse proxy.** ``client_ip``
+is sourced from ``request.remote_addr``, which Werkzeug fills from the
+*last hop* on the TCP connection. When the app sits behind a load
+balancer, ALB / ELB / nginx / Cloudflare, that hop is the proxy and
+every audit row records the proxy IP — not the originating client.
+For the audit trail to actually identify clients you must install
+Werkzeug's ``ProxyFix`` middleware (or an equivalent) so
+``X-Forwarded-For`` / ``Forwarded`` headers are honored:
+
+.. code-block:: python
+    :linenos:
+
+    from werkzeug.middleware.proxy_fix import ProxyFix
+
+    # ``x_for=1`` trusts exactly one X-Forwarded-For hop (your edge LB).
+    # If the request transits N reverse proxies you control end-to-end,
+    # raise this to N. Trusting too many hops lets clients spoof the IP.
+    app.wsgi_app = ProxyFix(app.wsgi_app, x_for=1, x_proto=1, x_host=1)
+
+Without ProxyFix (or a host-supplied equivalent), every ``client_ip``
+field in the audit stream is the LB's address and the audit trail
+loses most of its forensic value. The value of ``x_for`` is
+deployment-specific — adjust it for nested LB / CDN topologies, and
+**only** trust hops you operate.
+
+**Calling :func:`emit` from host code.** The ``emit`` function is part
+of the public surface: host integrations may import it and emit
+extra audit lines for non-CRUD operations they layer on top of the
+viewer (e.g. a custom admin route that bulk-tags objects). Usage:
+
+.. code-block:: python
+    :linenos:
+
+    from flask_s3_viewer.audit import emit as audit_emit
+    from flask_s3_viewer.auth import ACTION_LIST  # or ACTION_DOWNLOAD/...
+
+    # Call from inside a Flask request context so client_ip / user_agent
+    # are populated automatically; outside a request both fields emit as
+    # empty strings.
+    audit_emit(
+        action=ACTION_LIST,
+        namespace='my-bucket',
+        key='reports/2026/',   # caller pre-normalises (post-base_path)
+        user=current_user_email,
+        result='ok',
+        status_code=200,
+    )
+
+Prefer the ``flask_s3_viewer.auth.ACTION_*`` constants over raw
+strings; ``action`` and ``result`` are sanitised but the level mapping
+(``ok``→INFO, ``denied``→WARNING, ``error``→ERROR) depends on
+``result``. The signature is part of v1.x stability — additions will
+be backwards-compatible.
+
 Use Caching
 -----------
 S3 is charged per call. Therefore, Flask S3Viewer supports caching (currently only supports file caching, in-memory database will be supported later).

docs/doctrees/environment.pickle

@@ -53,6 +53,7 @@
 <li class="toctree-l1"><a class="reference internal" href="usage/templates.html"> Templates</a></li>
 <li class="toctree-l1"><a class="reference internal" href="license.html"> License</a></li>
 <li class="toctree-l1 current"><a class="current reference internal" href="#"> Changelog</a><ul>
+<li class="toctree-l2"><a class="reference internal" href="#unreleased">[1.1.0] - unreleased</a></li>
 <li class="toctree-l2"><a class="reference internal" href="#id2">1.0.1</a></li>
 <li class="toctree-l2"><a class="reference internal" href="#id4">1.0.0</a></li>
 <li class="toctree-l2"><a class="reference internal" href="#id5">1.0.0a1</a></li>
@@ -108,6 +109,61 @@
 <h1>Changelog<a class="headerlink" href="#changelog" title="Link to this heading"></a></h1>
 <p>All notable changes to this project will be documented in this file. Dates are displayed in UTC.</p>
 <p>Generated by <cite>``auto-changelog`</cite> &lt;<a class="reference external" href="https://github.com/CookPete/auto-changelog">https://github.com/CookPete/auto-changelog</a>&gt;`_.</p>
+<section id="unreleased">
+<h2>[1.1.0] - unreleased<a class="headerlink" href="#unreleased" title="Link to this heading"></a></h2>
+<blockquote>
+<div><p>Audit logging &amp; logging surface clean-up. The public API (<code class="docutils literal notranslate"><span class="pre">FlaskS3Viewer</span></code>
+constructor, <code class="docutils literal notranslate"><span class="pre">add_new_one</span></code>, <code class="docutils literal notranslate"><span class="pre">init_app</span></code>, <code class="docutils literal notranslate"><span class="pre">get_instance</span></code>,
+<code class="docutils literal notranslate"><span class="pre">get_boto_client</span></code>, <code class="docutils literal notranslate"><span class="pre">get_boto_session</span></code>) is unchanged — the changes
+below affect log output only. The version line is provisional;
+the final release tag is decided at release time.</p>
+</div></blockquote>
+<p><strong>Added</strong></p>
+<ul class="simple">
+<li><p>New audit logger <code class="docutils literal notranslate"><span class="pre">flask_s3_viewer.audit</span></code>. Every S3 CRUD action that
+flows through the blueprint (<code class="docutils literal notranslate"><span class="pre">list</span></code> / <code class="docutils literal notranslate"><span class="pre">download</span></code> / <code class="docutils literal notranslate"><span class="pre">upload</span></code> /
+<code class="docutils literal notranslate"><span class="pre">delete</span></code> / <code class="docutils literal notranslate"><span class="pre">presign</span></code>) emits exactly one structured record carrying
+<code class="docutils literal notranslate"><span class="pre">action</span></code>, <code class="docutils literal notranslate"><span class="pre">namespace</span></code>, <code class="docutils literal notranslate"><span class="pre">key</span></code>, <code class="docutils literal notranslate"><span class="pre">user</span></code>, <code class="docutils literal notranslate"><span class="pre">result</span></code>, <code class="docutils literal notranslate"><span class="pre">status_code</span></code>,
+<code class="docutils literal notranslate"><span class="pre">client_ip</span></code>, <code class="docutils literal notranslate"><span class="pre">user_agent</span></code>, and (on failure) <code class="docutils literal notranslate"><span class="pre">error</span></code>. Level mapping:
+<code class="docutils literal notranslate"><span class="pre">ok</span></code> → INFO, <code class="docutils literal notranslate"><span class="pre">denied</span></code> → WARNING, <code class="docutils literal notranslate"><span class="pre">error</span></code> → ERROR. Activation is
+pure-<code class="docutils literal notranslate"><span class="pre">logging</span></code>: attach a handler to <code class="docutils literal notranslate"><span class="pre">flask_s3_viewer.audit</span></code>, set its
+level, optionally pin <code class="docutils literal notranslate"><span class="pre">propagate</span> <span class="pre">=</span> <span class="pre">False</span></code>. No constructor flag.
+<code class="docutils literal notranslate"><span class="pre">flask_s3_viewer.audit.emit(...)</span></code> is a public helper host code may
+call to record extra non-CRUD operations on the same logger. Log
+injection is defended at this layer — ASCII control bytes in any
+user-controllable field (key / email / User-Agent / exception text)
+are escaped to <code class="docutils literal notranslate"><span class="pre">\xNN</span></code> before the record is built. UA is capped at
+256 bytes; free-form fields at 1024 bytes. See <em>Audit logging</em> in
+<code class="docutils literal notranslate"><span class="pre">docs/source/usage/configuration.rst</span></code> for ProxyFix guidance, JSON
+handler setup, and PII / ARN / bucket-name redaction filter
+examples.</p></li>
+</ul>
+<p><strong>Changed</strong></p>
+<ul class="simple">
+<li><p>Internal <code class="docutils literal notranslate"><span class="pre">logging.info(...)</span></code> / <code class="docutils literal notranslate"><span class="pre">logging.error(...)</span></code> /
+<code class="docutils literal notranslate"><span class="pre">logging.debug(...)</span></code> calls in <code class="docutils literal notranslate"><span class="pre">flask_s3_viewer/__init__.py</span></code>,
+<code class="docutils literal notranslate"><span class="pre">aws/s3.py</span></code>, <code class="docutils literal notranslate"><span class="pre">aws/session.py</span></code>, and <code class="docutils literal notranslate"><span class="pre">aws/cache.py</span></code> have been routed
+through per-module <code class="docutils literal notranslate"><span class="pre">logging.getLogger(__name__)</span></code> instances. Record
+<code class="docutils literal notranslate"><span class="pre">name</span></code> now reads <code class="docutils literal notranslate"><span class="pre">flask_s3_viewer.&lt;module&gt;</span></code> instead of <code class="docutils literal notranslate"><span class="pre">root</span></code>. Host
+pipelines that filter by record <code class="docutils literal notranslate"><span class="pre">name</span></code> (rather than by handler
+attachment) need to update their allow-lists; host pipelines that
+attach handlers to the root logger (or to <code class="docutils literal notranslate"><span class="pre">flask_s3_viewer</span></code>) are
+unaffected — the module loggers default to <code class="docutils literal notranslate"><span class="pre">propagate=True</span></code>.</p></li>
+<li><p><code class="docutils literal notranslate"><span class="pre">app.url_map</span></code> registration dump was demoted from INFO to DEBUG so
+registering many namespaces (or large blueprints) no longer floods
+INFO logs at app startup.</p></li>
+</ul>
+<p><strong>Fixed</strong></p>
+<ul class="simple">
+<li><p><code class="docutils literal notranslate"><span class="pre">aws/s3.py</span></code>‘s <code class="docutils literal notranslate"><span class="pre">PURGE:</span></code> / <code class="docutils literal notranslate"><span class="pre">MKDIR:</span></code> / <code class="docutils literal notranslate"><span class="pre">UP_OBJECT:</span></code> DEBUG lines used
+the broken <code class="docutils literal notranslate"><span class="pre">logging.debug('PURGE:',</span> <span class="pre">name)</span></code> form (the second
+positional argument was treated as a logging-args tuple, against an
+unparameterised message string, which the stdlib silently drops).
+They now use <code class="docutils literal notranslate"><span class="pre">logger.debug('PURGE:</span> <span class="pre">%s',</span> <span class="pre">name)</span></code> etc., so a host that
+pins the logger at DEBUG actually sees the keys involved in each
+cache invalidation / placeholder create / upload event.</p></li>
+</ul>
+</section>
 <section id="id2">
 <h2><a class="reference external" href="https://github.com/hidekuma/flask-s3-viewer/compare/1.0.0...1.0.1">1.0.1</a><a class="headerlink" href="#id2" title="Link to this heading"></a></h2>
 <blockquote>