TL;DR: Apologies for the long post. In a nutshell I am trying to debug a CSP report-uri. If I am missing critical information please let me know.
CSP implementation: Flask-Talisman
The attribute that needs to be set: content_security_policy_report_uri
There does not seem to be a lot of information out there on how to capture this report
I can't find anything specific in the Flask-Talisman documentation
As Flask-Talisman only sets headers, including the report-uri, I imagine this is outside the scope of the extension anyway
The route
All resources I've found have roughly the same function:
https://www.merixstudio.com/blog/content-security-policy-flask-and-django-part-2/
http://csplite.com/csp260/
https://github.com/GoogleCloudPlatform/flask-talisman/issues/21
The only really detailed explanation I've found for this route is below (it is not related to Flask-Talisman however)
From https://www.merixstudio.com/blog/content-security-policy-flask-and-django-part-2/ (This is what I am currently using)
# app/routes/report.py
import json
import pprint
from flask import request, make_response, Blueprint
...
bp = Blueprint("report", __name__, url_prefix="report")
...
@bp.route('/csp-violations', methods=['POST'])
def report():
"""Receive a post request containing csp-resport.
This is the report-uri. Print report to console and
return Response object.
:return: Flask Response object.
"""
pprint.pprint(json.loads(str(request.data, 'utf-8')))
response = make_response()
response.status_code = 200
return response
...
This route only receives a 400 error and I am at a loss in finding out how to actually debug this
127.0.0.1 - - [04/Nov/2021 14:29:09] "POST /report/csp-violations HTTP/1.1" 400 -
I have tried with a GET request and could see I am receiving an empty request, which seems to mean that the CSP report isn't being delivered (response from https://127.0.0.1:5000/report/csp-violations)
# app/routes/report.py
...
@bp.route('/csp-violations', methods=['GET', 'POST'])
def report():
...
b''
b''
127.0.0.1 - - [04/Nov/2021 18:03:52] "GET /report/csp_violations HTTP/1.1" 200 -
Edit: Definitely receiving nothing
...
@bp.route("/csp_violations", methods=["GET", "POST"])
def report():
...
return str(request.args) # ImmutableMultiDict([ ])
Will not work with a GET or POST request (still 400)
...
@bp.route("/csp_violations", methods=["GET", "POST"])
def report():
content = request.get_json(force=True)
...
Without force=True
@bp.route("/csp_violations", methods=["GET", "POST"])
def report():
content = request.get_json() # json.decoder.JSONDecodeError
...
Chromium (same result on Chrome, Brave, and FireFox)
When I check this out on Chromium under CTRL-SHIFT-I > Network I see
Request URL: https://127.0.0.1:5000/report/csp_violations
Request Method: POST
Status Code: 400 BAD REQUEST
Remote Address: 127.0.0.1:5000
Referrer Policy: strict-origin-when-cross-origin
But apparently the payload does exist at the bottom under "Request Payload"...
{
"document-uri": "https://127.0.0.1:5000/",
"referrer": "",
"violated-directive": "script-src-elem",
"effective-directive": "script-src-elem",
"original-policy": "default-src 'self' https://cdnjs.cloudflare.com https://cdn.cloudflare.com https://cdn.jsdelivr.net https://gravatar.com jquery.js; report-uri /report/csp-violations",
"disposition": "enforce",
"blocked-uri": "inline",
"line-number": 319,
"source-file": "https://127.0.0.1:5000/",
"status-code": 200,
"script-sample": ""
}
Is the CSP blocking this request?
After reading Python Flask 400 Bad Request Error Every Request I set all requests to HTTPS with the help of https://stackoverflow.com/a/63708394/13316671
This fixed a few unittests but no change to the 400 error for this particular route
# app/config.py
from environs import Env
from flask import Flask
env = Env()
env.read_env()
class Config:
"""Load environment."""
...
@property
def PREFERRED_URL_SCHEME(self) -> str: # noqa
return env.str("PREFERRED_URL_SCHEME", default="https")
...
# app/__init__.py
from app.config import Config
def create_app() -> Flask:
app = Flask(__name__)
app.config.from_object(Config())
...
return app
flask run --cert="$HOME/.openssl/cert.pem" --key="$HOME/.openssl/key.pem"
400 Bad Request
Through what I've read a 400 Bad Request error usually happens from an empty request or form
https://stackoverflow.com/a/14113958/13316671
...the issue is that Flask raises an HTTP error when it fails to find a key in
the args and form dictionaries. What Flask assumes by default is that if you
are asking for a particular key and it's not there then something got
left out of the request and the entire request is invalid.
https://stackoverflow.com/a/37017020/13316671
99% of the time, this error is a key error caused by your requesting a key in
the request.form dictionary that does not exist. To debug it, run
print(request.form)
request.data in my case
Through trying to fix this I have gone down the rabbit-hole in regard to just seeing the cause of the 400 error
I have implemented the following
https://stackoverflow.com/a/34172382/13316671
import traceback
from flask import Flask
...
app = Flask(__name__)
...
@app.errorhandler(400)
def internal_error(exception): # noqa
print("400 error caught")
print(traceback.format_exc())
And added the following to my config
I haven't checked, but I think these values already get set alongside DEBUG
# app/config.py
from environs import Env
env = Env()
env.read_env()
class Config:
"""Load environment."""
...
@property
def TRAP_HTTP_EXCEPTIONS(self) -> bool: # noqa
"""Report traceback on error."""
return env.bool("TRAP_HTTP_EXCEPTIONS", default=self.DEBUG) # noqa
@property
def TRAP_BAD_REQUEST_ERRORS(self) -> bool: # noqa
"""Report 400 traceback on error."""
return env.bool("TRAP_BAD_REQUEST_ERRORS", default=self.DEBUG) # noqa
...
I still am not getting a traceback. The only time werkzeug shows a traceback is through a complete crash, for something like a syntax error
It seems, because I am not initializing this request that the app continues to run through the 400 code, no problem
Summary
Main conclusion I'm drawing is that for some reason the report-uri is not valid because I can see the payload exists, I am just not receiving it
I used the relative route as the subdomain would be different for a localhost and a remote. It looks as though the request is being made to the full URL in the Chromium snippet though.
Could I be receiving invalid headers https://stackoverflow.com/a/45682197? If so how can I debug that?
https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Security-Policy/report-uri
Note: The report-uri is deprecated, but I don't think most browsers support the report-to parameter
EDIT:
I have taken the following out of my application factory (this code renders my exceptions - which I posted as an answer here so it is available https://stackoverflow.com/a/69723406/13316671
...
app = Flask(__name__)
config.init_app(app)
...
# exceptions.init_app(app)
...
return app
After doing this (on FireFox Private mode), I made another POST (I've removed the GET method) I can see the response - here I have managed to collect a Werkzeug traceback finally:
wtforms.validators.ValidationError: The CSRF token is missing.
Working out how to authenticate a csp-report coming from the browser
So far I've had a look at this: Add a new Http Header in Content-Security-Policy-Report-Only "report-uri" POST call
CodePudding user response：
Try this piece of code:
@app.route('/report-csp-violations', methods=['POST'])
def report():
content = request.get_json(force=True) # that's where the shoe pinches
print(json.dumps(content, indent=4, sort_keys=True))
response = make_response()
response.status_code = 204
return response
I think that request.data tries automatically parse JSON data and for success parsing it expects application/json MIME type been sent.
But violation reports is sent with the application/csp-report MIME type, therefore Flask considers it as a wrong data from the client -> 404 Bad Request.
.get_json(force=True) means ignore the mimetype and always try to parse JSON.
Also I don't think you need to do a conversion to utf-8 because according to rfc4627 "3. Encoding":
JSON text SHALL be encoded in Unicode. The default encoding is UTF-8.
CodePudding user response：
For now I have exempted the view from CRSFProtect.
I think that should be fine as this view does not return a template, so I haven't been able to add
<form method="post">
{{ form.csrf_token }}
</form>
And the below didn't work for me (Don't think it's a valid token)
# app/routes/report.py
...
def report():
response = make_response()
response.headers["X-CSRFToken"] = csrf.generate_csrf()
...
return response
...
With the instantiated CSRFProtect object...
# app/extensions.py
...
from flask_wtf.csrf import CSRFProtect
...
csrf_protect = CSRFProtect()
...
def init_app(app: Flask) -> None:
...
csrf_protect.init_app(app)
...
...
...decorate the view
# app/routes/report.py
from app.extensions import csrf_protect
...
@blueprint.route("/csp_violations", methods=["POST"])
@csrf_protect.exempt
def report():
....
127.0.0.1 - - [06/Nov/2021 21:30:46] "POST /report/csp_violations HTTP/1.1" 204 -
{
"csp-report": {
"blocked-uri": "inline",
"column-number": 8118,
"document-uri": "https://127.0.0.1:5000/",
"line-number": 3,
"original-policy": "default-src" ...
"referrer": "",
"source-file": ...
}
}
I had a system going where my errors were obfuscated, so that's my mistake. This is not the first time I've had trouble with CSRFProtect.
I've fixed the debugging problem with
# app/exceptions.py
...
def init_app(app: Flask) -> None:
...
def render_error(error: HTTPException) -> Tuple[str, int]:
# the below code is new
app.logger.error(error.description)
...
...
So, this is what I would see now if I was still getting this error
[2021-11-06 21:23:56,054] ERROR in exceptions: The CSRF token is missing.
127.0.0.1 - - [06/Nov/2021 21:23:56] "POST /report/csp_violations HTTP/1.1" 400 -