1. Overview
Client URL (cURL) is a command line utility in Linux that supports data exchange between client and server via many protocols, including HTTP and HTTPS. In this tutorial, we’ll learn how to use cURL for sending raw body data over a POST request.
2. Understanding –data and –data-raw
The curl command supports the –data and –data-raw options to transfer data over POST requests. In this section, let’s understand the default behavior of using curl with these options.
First, we need a server endpoint that would accept the requests originating from our end as a client. So, let’s visit webhook.site and get the server endpoint:
https://webhook.site/5610141b-bd31-4940-9a83-1db44ff2283e
Now, let’s initialize the variable website with this endpoint for reusability purposes:
$ website="https://webhook.site/5610141b-bd31-4940-9a83-1db44ff2283e"
Next, let’s use the curl command with –data and –data-raw options to send text over a POST request:
$ curl --data "simple_body" --trace-ascii website-data.log "$website"
$ curl --data-raw "simple_body" --trace-ascii website-data-raw.log "$website"
We must note that we used the –trace-ascii option to trace the requests and capture the trace logs in the website-data.log and website-data-raw.log files.
Continuing further, let’s inspect the value of the Content-Type header by doing a grep over the captured trace logs:
$ grep --max-count=1 Content-Type website-data-raw.log website-data.log
website-data-raw.log:0083: Content-Type: application/x-www-form-urlencoded
website-data.log:0083: Content-Type: application/x-www-form-urlencoded
We can infer that even though we didn’t specify the Content-Type in the original requests, curl sets it to the default value of application/x-www-form-urlencoded. As a result, the requests didn’t send the text “simple_body” as the raw data but as form data.
Finally, let’s confirm this visually through the webhook.site page:
We can notice that “simple_body” is present as a key with an empty value in the Form values section.
In this section, we’ll use the –header option to set the Content-Type header explicitly so that curl doesn’t make the default choice.
Let’s go ahead and make the curl requests with the same content but with the correct headers:
$ curl --header "Content-Type: text/plain" --data "simple_body" --trace-ascii website-data.log "$website"
$ curl --header "Content-Type: text/plain" --data-raw "simple_body" --trace-ascii website-data-raw.log "$website"
Next, let’s inspect the trace logs to verify the Content-Type value in the request headers:
$ grep --max-count=1 Content-Type website-data-raw.log website-data.log
website-data-raw.log:006f: Content-Type: text/plain
website-data.log:006f: Content-Type: text/plain
We can notice that the Content-Type header is correct. Further, we shall see that we used the grep with the –max-count=1 to limit the output result to the request header.
Finally, let’s also confirm it visually by visiting the webhook.site page:
We can see that raw text is present in the Raw content section, and the Form values section is empty now.
4. Sending Content from File
At times, the request body might contain a lot of text, and it’s recommended that we send the content directly from a file.
First, let’s store the content in a content.txt file:
$ echo "simple_body" > content.txt
We must note that we’re adding short text in the file for simplicity, and the same approach applies to larger files.
Next, we need to understand that curl supports sending the raw text via files using the –data option using the @file notation but not with the –data-raw option. So let’s go ahead and make the curl request using the –data option with the @file notation:
$ curl --header "Content-Type: plain/text" --data @content.txt --trace-ascii website-data.log "$website
Further, let’s verify by inspecting the website-data.log trace file for the request body:
$ grep -B1 -i simple_body website-data.log
=> Send data, 11 bytes (0xb)
0000: simple_body
Finally, let’s also see what happens when we use @file with the –data-raw option:
$ curl --header "Content-Type: plain/text" --data-raw @content.txt --trace-ascii website-data-raw.log "$website"
$ grep -B1 -i content.txt website-data-raw.log
=> Send data, 12 bytes (0xc)
0000: @content.txt
We can notice that this time, curl didn’t understand that it needed to fetch the content from the content.txt file. Instead, it sent the text “@content.txt” as the request body.
5. Conclusion
In this article, we learned how to use the curl command for sending raw body content over POST requests. Additionally, we also learned to send large content using files.
