Introduction

In this tutorial, I will show how to search the full archive using the curl request. All curl commands need to be run at the command line. The Terminal window inside R studio will work for this.

Step 1: Create a Bearer Token

Navigate to your twitter developer account and access the app that you created. Navigate to the “Keys and Tokens” page and locate the “Consumer API Keys.” These keys will be used to generate your bearer token. Enter these keys into the following curl request.

curl -u '<API key>:<API secret key>' \
   --data 'grant_type=client_credentials' \
   'https://api.twitter.com/oauth2/token'

Here’s an example of what it should look like (these aren’t real keys)

curl -u 'j6ug2DsN2nDImNevwSyPhjz6rd:xrdUZzMpmwTPIdnNnDIcxogUgRmZevwfSyPD6KrkhSjaXX32zT4z8' \
  --data 'grant_type=client_credentials' \
  'https://api.twitter.com/oauth2/token'

note: The backslashes represent new lines (only for formatting in this document. When you really use these commands, you can do it all on 1 line, but you need to remove all the backslashes)

You will get a response that holds the bearer token that we will be using. For example:

{"token_type":"bearer","access_token":"AAAAAAAAAAAAAAAAAAAAAMLheAAAAAAA0%2BuSeid%2BULvsea4JtiGRiSDSJSI%3DEUifiRBkKG5E2XzMDjRfl76ZC9Ub0wnz4XsNiRVBChTYbJcE3F"}

Copy the code that is returned:

AAAAAAAAAAAAAAAAAAAAAMLheAAAAAAA0%2BuSeid%2BULvsea4JtiGRiSDSJSI%3DEUifiRBkKG5E2XzMDjRfl76ZC9Ub0wnz4XsNiRVBChTYbJcE3F

Step 2: Write the request

Now that we have our bearer token, we need to use it in the main curl request which looks like this:

curl --request POST \
  --url https://api.twitter.com/1.1/tweets/search/fullarchive/<ENV>.json \
  --header 'authorization: Bearer <BEARER_TOKEN>' \
  --header 'content-type: application/json' \
  --data '{
                "query":"from:TwitterDev lang:en",
                "maxResults": "100",
                "fromDate":"<YYYYMMDDHHmm>", 
                "toDate":"<YYYYMMDDHHmm>"
                }'

So, once we insert our <BEARER_TOKEN>, and developer environment <ENV> it would look something like this:

curl --request POST --url https://api.twitter.com/1.1/tweets/search/fullarchive/datasci485App.json --header 'authorization: Bearer AAAAAAAAAAAAAAAAAAAAADVsCQEAAAAA4bvDPyYPZ%2BBRMBQ0ZBaGQn0n9KQ%3DBZzFA3DJDYgKHbPJIqnkl5LqkNLEppCfdf19xclFjhHmMZKQuV' --header 'content-type: application/json' --data '{"query":"from:TwitterDev lang:en", "maxResults":"500", "fromDate":"202002010000", "toDate":"202002040000"}'

Step 3: Write the results to a file

Specify the file name (path optional) at the end of this request as > filename.json. No quotes around the path name.

curl --request POST --url https://api.twitter.com/1.1/tweets/search/fullarchive/datasci485App.json --header 'authorization: Bearer AAAAAAAAAAAAAAAAAAAAADVsCQEAAAAA4bvDPyYPZ%2BBRMBQ0ZBaGQn0n9KQ%3DBZzFA3DJDYgKHbPJIqnkl5LqkNLEppCfdf19xclFjhHmMZKQuV' --header 'content-type: application/json' --data '{"query":"from:TwitterDev lang:en", "maxResults":"500", "fromDate":"202002010000", "toDate":"202002040000"}' > tweet_block_1.json