Tips for building a Twitter bot with R and Github Actions

16 January 2022

UPDATE 28 January 2022: It seems one indeed does need to apply for Twitter Developer elevated API access to use the rtweet v1.1 endpoints. I’ve updated this post to reflect this.

UPDATE 10th May 2022 Added the step on changing your app to read/write permissions and regenerating your tokens.

Important: Start this process at least 48 hours ahead of the time you plan to code the bot, as it will take up to 48 hours for your Twitter Elevated API access to be approved.

A couple of weeks ago I created my very first twitter bot which was immensely satisfying. It was also quite a frustrating process and this post should save you a lot of frustration :). You can view the code in the github repo.

I can’t give enough thanks to Matt Dray for the “A Twitter bot with {rtweet} and GitHub Actions” post which I leaned on heavily to get me very far. I suggest you read it and follow it.

In this post I’ve added a few tips to help make your bot’s app development that tiny bit smoother.

Creating the bot’s Twitter Account using your own Gmail address

If you’re posting on your personal timeline, then no need to do this. If you want to create a twitter account for your bot, then you’ll need a separate email address from the one you’ve used for your own account.

What I did (in order to avoid setting up a new email account just for this bot to have a twitter account) was to use a handy feature of gmail. You can append any gmail address with whatever you want, and those emails will be sent directly to the main gmail account.

For example, let’s say your personal address is myemail@gmail.com, then you can create a new twitter account using myemail+[whatever]@gmail.com. Twitter picks it up as a distinct address and allows you to create a new account with it and all verification messages get forwarded to myemail@gmail.com. Pretty neat. One less email account you need to manage.

Add your phone number

Once you’ve created your new account, you’ll need to log into it and find the settings page, currently by clicking the 3 dots on the lower left of the home screen and then selecting Your Account, and then Account information. Enter your phone number and you’ll be asked for verification number via SMS. This is needed because you can’t create an app in the Developer Portal without it.

Apply for Elevated API access

Twitter recently switched to APIv2 as its main API. The rtweet package uses APIv1.1 endpoints. This means that you need to apply for Elevated API access. This can take up to 48 hours to be approved, so create your account ahead of time and apply for the access.

You need to fill out a form which describes what you want to use the API for.

Edit your app from read only to read/write permissions

Once you’ve been granted Elevated Access, you’ll need to update your app permissions and then regenerate the app tokens.

Head over to the gear icon on your listed app:

Next, go to Edit your User Authentication Settings:

In the settings, you can disable OAuth2.0 (it’s for v2 endpoints only), enable OAuth1.0a, disable request email from users and then change the permission to read/write, or read/write, direct message depending on whether you need Direct Message or not. For most bots I would advise just use Read/Write.

For all the remaining authentication settings, which are where your privacy policy etc lives, I just add my blog about page to all of them.

Once you’ve done all that and saved, then you need to go back to your Keys and Tokens, and regenerate your Access Token and Access Secret – you should see that this screen confirms that you now have Read and Write permissions.

Consider using a public Github repo rather than a private one

The reason for this is that billing limits apply to private repo’s i.e. you have a maximum limit of 2000 minutes of GH action runtime under a private repo. That may sound like a lot, especially as (for example) my bot takes just 2-3 minutes of time to run. However, different time multipliers apply depending on the Operating System you invoke. macOS for example, has a 10x multiplier.

These time limits only apply to private repos, not to public ones. Why Github doesn’t have limits on the public ones , I’m not sure. Surely some sort of fair usage or throttling applies somewhere, but I’ve not found any mention of this.

Rename your secrets and tokens

All tutorials leave the default secret names at TWITTER_* but I suggest renaming your keys, secrets and tokens so that if you wish to test multiple bots locally, you can do so. For mine I just added “RBOT_” infront of each one. That way, when I make future bots, I can keep all the tokens and secrets in one .renviron folder locally.

rbot_token <- rtweet::create_token(
  app = "BigBookofR",
  # the name of the Twitter app
  consumer_key = Sys.getenv("RBOT_TWITTER_CONSUMER_API_KEY"),
  consumer_secret = Sys.getenv("RBOT_TWITTER_CONSUMER_API_SECRET"),
  access_token = Sys.getenv("RBOT_TWITTER_ACCESS_TOKEN"),
  access_secret = Sys.getenv("RBOT_TWITTER_ACCESS_TOKEN_SECRET"),
  set_renv = FALSE
)

Bonus tip, set set_renv = FALSE – this prevents rwteet from creating a local token file every time the bot runs locally – I did a lot of test runs and just prefer not to have a new token file saved every time.

Save secrets in Repository Secrets

In all posts I had read, they all say to store your secret keys in Github Repo under Settings > Secrets. What’s changed though is that there are now 2 types of Secrets you can store i.e. Environment secrets and Repository secrets. I wasn’t sure which would be the same as the older posts showed when there wasn’t this distinction.

I tried both, with all sorts of combinations of how the secrets should be retrieved in the YAML file and Script. It of course didn’t helped that I pasted the wrong secret under the wrong name (🤦‍♂️) but this confused me enough that I only triple checked that my secrets were correct. On the 4th check of my secrets I pasted them correctly :/.

Long story short, you paste them in the Repository secrets.

Alternative way pass the bot app credentials

Now this might not be a needed fix, but if you’re having trouble with passing the app credentials remotely, try the setup I used which differs slightly from Matt’s setup. I created the app token, then then passed in with the tweet as follows

# Send tweet --------------------------------------------------------------

# Create a token containing your Twitter keys
rbot_token <- rtweet::create_token(
  app = "BigBookofR",
  # the name of the Twitter app
  consumer_key = Sys.getenv("RBOT_TWITTER_CONSUMER_API_KEY"),
  consumer_secret = Sys.getenv("RBOT_TWITTER_CONSUMER_API_SECRET"),
  access_token = Sys.getenv("RBOT_TWITTER_ACCESS_TOKEN"),
  access_secret = Sys.getenv("RBOT_TWITTER_ACCESS_TOKEN_SECRET"),
  set_renv = FALSE
)

# Example: post a tweet via the API
rtweet::post_tweet(status = book_status,
                   token = rbot_token)

Test using “On Push” in the YAML

After testing the app locally, you’ll of course want to test it remotely and see if the Github action triggers. The problem with running a scheduled action is you need to wait for your schedule (in my case every few hours).

You can edit the action trigger using CRON (the scheduler) to rather being “On push” which means that every time you push to the repo, the action will be triggered. I then comment out the “On Push” and revert back to the CRON scheduler when I am done testing. This is all specified in the YAML file.

name: bigbookofr_bot
on:
  #schedule:
  #  - cron: '0 */3 * * *'
  push:
    branches:
      - main

There’s lot’s of other fancy options if you want only specific branches to trigger or a whole lot of other configurations.

In my RScript, I also just use this commented line that I edit slightly if I want to trigger a diff to commit and push, without editing any other code.

# Comment to trigger diff for testing.

Using a CRON expression generator

If you’ve never worked with CRON before the expressions can be really confusing. You can check that the schedule is correctly described by using somethign like CRON Expression Generator or Crontab guru.

On build: MacOS vs others – what’s the difference?

One of the things your YAML file specifies is which OS to use.

jobs:
  bigbookofr_bot-post:
    runs-on: macOS-latest

I suggest just sticking to macOS. Other options are Windows Server or Ubuntu.

I tested the other two, and both failed and had longer run times. Whether the longer run times were impacted by the fails I am not sure, but for reference, both failed when in came to loading or using the {googlesheets4} package which I needed to retrieve the data I wanted to post.

The macOS action runs in 2-3 minutes for my bot. For windows it was about 10 mins and failed, and for Ubuntu was 20 mins before failing.

That’s it folks! I hope this helps :).

---

If you liked that post, here’s some related posts for you:

A new way to discover R Programming books!

---

Newsletter subscribers get a free copy of Project Management Fundamentals for Data Analysts worth $12.

Once you’ve subscribed, you’ll get a follow up email with a link to your free copy.