Invalid CSRF Token when Uploading File via REST API

Moved Technical Support
  • I have been using the REST API to write a tool to import data, and have had good success up to the point where I want to upload images to a post, or just upload images in general. I have tried both the /api/admin/upload/file and the /api/post/upload endpoints, but no matter what combination of headers I seem to try, I am getting the following errors on the server:

    2022-08-15T12:35:47.558Z [4567/165] - error: POST /api/admin/upload/file?_uid=1
    invalid csrf token
    2022-08-15T12:38:16.394Z [4567/165] - error: POST /api/post/upload?_uid=1
    invalid csrf token

    I am using the Python Requests library to access the REST API, and my logic is as follows:

    • Create a Requests session
    • Attach the generated Bearer token to the session
    • Access the /api/config endpoint to retrieve the csrf_token
    • Attach retrieved csrf_token to the session header for key x-csrf-token
    • Post to the /api/admin/upload/file?_uid=1 endpoint, setting the files[] param to a Python file pointer

    The attempts to post to the /api/post/upload are similar, but I am also attaching a cid to the post data for a valid category. Either way, I get the same 403 Client Error: Forbidden for url ... error in the response, and the invalid csrf token on the server.

    This is what a raw request (output from Python Requests with debugging turned on) looks like when I debug the POST:

    send: b'POST /api/post/upload?_uid=1 HTTP/1.1\r\nHost: localhost:4567\r\nUser-Agent: python-requests/2.28.1\r\nAccept-Encoding: gzip, deflate\r\nAccept: */*\r\nConnection: keep-alive\r\nAuthorization: Bearer 8bab7c95-da3c-44a7-926c-750158b7f6eb\r\nx-csrf-token: crGjXcZT-unxVCdPwBPmIEoFMgGCvyhGuI44\r\nCookie: express.sid=s%3AXhIQf0bWxYFViKKID5OqOobbG5dRAx7W.sgxJnGcULUbcQZnEZJfeFMMK%2FDiX0m8RIEYkAR%2BKgzQ\r\nContent-Length: 254\r\nContent-Type: multipart/form-data; boundary=aef166f35592d3b4beed61b08ccdf152\r\n\r\n'
    send: b'--aef166f35592d3b4beed61b08ccdf152\r\nContent-Disposition: form-data; name="cid"\r\n\r\n12\r\n--aef166f35592d3b4beed61b08ccdf152\r\nContent-Disposition: form-data; name="files[]"; filename="1659448063198-foo.txt"\r\n\r\ntest me\n\r\n--aef166f35592d3b4beed61b08ccdf152--\r\n'
    reply: 'HTTP/1.1 403 Forbidden\r\n'
    header: Cross-Origin-Opener-Policy: same-origin
    header: Cross-Origin-Resource-Policy: same-origin
    header: X-DNS-Prefetch-Control: off
    header: Expect-CT: max-age=0
    header: X-Frame-Options: SAMEORIGIN
    header: Strict-Transport-Security: max-age=15552000; includeSubDomains
    header: X-Download-Options: noopen
    header: X-Content-Type-Options: nosniff
    header: Origin-Agent-Cluster: ?1
    header: X-Permitted-Cross-Domain-Policies: none
    header: Referrer-Policy: strict-origin-when-cross-origin
    header: X-XSS-Protection: 0
    header: X-Powered-By: NodeBB
    header: Content-Security-Policy: frame-ancestors 'self'
    header: Content-Type: text/plain; charset=utf-8
    header: Content-Length: 9
    header: ETag: W/"9-PatfYBLj4Um1qTm5zrukoLhNyPU"
    header: Set-Cookie: express.sid=s%3AXhIQf0bWxYFViKKID5OqOobbG5dRAx7W.sgxJnGcULUbcQZnEZJfeFMMK%2FDiX0m8RIEYkAR%2BKgzQ; Path=/; HttpOnly; SameSite=Lax
    header: Date: Mon, 15 Aug 2022 12:41:43 GMT
    header: Connection: keep-alive
    header: Keep-Alive: timeout=5

    I have replicated the process in the browser with the Dev Tools open, to see what headers I might be missing, but other than a session ID, I am not seeing the difference, but it is clearly working through the web application. I also tried to step through the tests for uploading an image to see if I was missing something, and other than the cookies not completely matching, nothing jumped out at me there.

    Any help here is appreciated, as it seems this is last piece I need working for my import process.

  • Maybe you can debug on the server side and check if x-csrf-token is in the correct place. We use the csurf module and you should be able put a breakpoint at in node_modules/csurf/index.js in your nodebb folder. The module looks for the token in a few different places as you can see here

  • It looks like you may be missing cookies? You probably need to add a jar to your request session.

    Might help to reference the uploads tests:

  • Thanks for the quick replies.

    I tried setting the csrf token in a couple different ways, per @baris suggestion, but didn't get any different results. Thanks for the pointer to the code that reads the tokens in different ways, that was a good thought.

    My suspicion is that @PitaJ is on to something with the cookie jar, I did notice that when I looked through the test code before. I haven't had too much time to spend on this, but did attempt to do some basic cookie jar tests with Requests, and I am still getting the same error.

    I hope to have more time later this week to work through this further, and will post any updates and relevant code here if I get something working or have more questions.

  • @Jeremy-Snyder It may be helpful to see how we do it in our testing suite 🙊

  • Finally getting back around to testing this, still not having much luck. After digging through a lot of the code, and reading through the unit tests for the /api/post/upload endpoint, a user is logged in first.

    I have been trying to do this with only an API token, as this process is loading data from another system. I really don't want to have to store a password in addition to an API token when doing my data load.

    I guess my next question is: Is /api/post/upload even usable without logging in with a username/password?

  • julianJ julian moved this topic from NodeBB Development on

Suggested Topics