Twitter bot playground

How to build and deploy a multifunctional Twitter bot!

This is a reference for me and anyone else that's interested in Twitter bots in JavaScript.

All of the examples here use the npm package twit.

We'll go through setting up a simple bot so each of these examples can be run with it.

I'm going to assume that you have nodejs installed along with npm and that you are comfortable with the terminal.

If you are not familiar node or do not have your environment set up to use it take a look at the README.md on my Twitter bot bootstrap repo which details getting a Twitter application set up and a development environment with c9.

A great resource is Aman Mittal's Awesome Twitter bots repo which has resources and bot examples.

A lot of this information is already out there I'm hoping this is all the information someone will need to get started with their own Twitter bot. I'm doing this for my own learning and hopefully other people will get something out of this as well.

Set up the bot

Before touching the terminal or writing any code we'll need to create a Twitter app to get our API keys, we'll need them all:

1Consumer Key (API Key)
2Consumer Secret (API Secret)
3Access Token
4Access Token Secret

Keep the keys somewhere safe so you can use them again when you need them, we're going to be using them in the .env file we're going to create.

We're using dotenv so that if at some point in the future we want to add our bot to GitHub the Twitter API keys are not added to GitHub for all to see.

Starting from scratch, create a new folder via the terminal and initialise the package.json via npm or yarn we'll need twit and dotenv for all these examples.

I'll be using yarn for all these examples, you can use npm if you prefer.

Terminal commands:

1mkdir tweebot-play
2cd tweebot-play
3yarn init -y
4yarn add twit dotenv
5touch .env .gitignore index.js

If you take a look at the package.json that was created it should look something like this:

1{
2  "name": "tweebot-play",
3  "version": "1.0.0",
4  "main": "index.js",
5  "author": "Scott Spence <spences10apps@gmail.com> (https://spences10.github.io/)",
6  "license": "MIT",
7  "dependencies": {
8    "dotenv": "^4.0.0",
9    "twit": "^2.2.5"
10  }
11}

Add an npm script to the package.json to kick off the bot when we're testing and looking for output:

1"scripts": {
2    "start": "node index.js"
3  },

It should look something like this now:

1{
2  "name": "tweebot-play",
3  "version": "1.0.0",
4  "main": "index.js",
5  "scripts": {
6    "start": "node index.js"
7  },
8  "author": "Scott Spence <spences10apps@gmail.com> (https://spences10.github.io/)",
9  "license": "MIT",
10  "dependencies": {
11    "dotenv": "^4.0.0",
12    "twit": "^2.2.5"
13  }
14}

Now we can add the following pointer to the bot in index.js, like so:

1require('./src/bot')

So when we use yarn start to run the bot it calls the index.js file which runs the bot.js file from the src folder we're going to create.

Now we add our API keys to the .env file, it should look something like this:

1CONSUMER_KEY=AmMSbxxxxxxxxxxNh4BcdMhxg
2CONSUMER_SECRET=eQUfMrHbtlxxxxxxxxxxkFNNj1H107xxxxxxxxxx6CZH0fjymV
3ACCESS_TOKEN=7xxxxx492-uEcacdl7HJxxxxxxxxxxecKpi90bFhdsGG2N7iII
4ACCESS_TOKEN_SECRET=77vGPTt20xxxxxxxxxxxZAU8wxxxxxxxxxx0PhOo43cGO

In the .gitignore file we need to add .env and node_modules

1# Dependency directories
2node_modules
3
4# env files
5.env

Then init git:

1git init

Ok, now we can start to configure the bot, we'll need a src folder a bot.js file and a config.js file.

Terminal:

1mkdir src
2cd src
3touch config.js bot.js

Then we can set up the bot config, open the config.js file and add the following:

1require('dotenv').config()
2
3module.exports = {
4  consumer_key: process.env.CONSUMER_KEY,
5  consumer_secret: process.env.CONSUMER_SECRET,
6  access_token: process.env.ACCESS_TOKEN,
7  access_token_secret: process.env.ACCESS_TOKEN_SECRET,
8}

Ok, that's the bot config done now we can set up the bot, each of the examples detailed here will have the same three lines of code:

1const Twit = require('twit')
2const config = require('./config')
3
4const bot = new Twit(config)

Ok, that's it out bot is ready to go, do a test with yarn start from the terminal, we should get this for output:

1yarn start
2yarn start v0.23.4
3$ node index.js
4Done in 0.64s.

Bot is now configured and ready to go!🚀

Post Statuses

Firstly post statuses, with .post('statuses/update'... bot will post a hello world! status.

1bot.post(
2  'statuses/update',
3  {
4    status: 'hello world!',
5  },
6  (err, data, response) => {
7    if (err) {
8      console.log(err)
9    } else {
10      console.log(`${data.text} tweeted!`)
11    }
12  }
13)

Work with users

To get a list of followers ids use .get('followers/ids'... and include the account that you want the followers of, in this example we're using @DroidScott, you can use any account you like. We can then log them out to the console in this example.

1bot.get(
2  'followers/ids',
3  {
4    screen_name: 'DroidScott',
5    count: 5,
6  },
7  (err, data, response) => {
8    if (err) {
9      console.log(err)
10    } else {
11      console.log(data)
12    }
13  }
14)

You can specify with the count parameter how many results you get up to 100 at a time.

Or to get a detailed list you can use .get('followers/list'...

Here we print off a list of user.screen_name's up to 200 per call.

1bot.get(
2  'followers/list',
3  {
4    screen_name: 'DroidScott',
5    count: 200,
6  },
7  (err, data, response) => {
8    if (err) {
9      console.log(err)
10    } else {
11      data.users.forEach(user => {
12        console.log(user.screen_name)
13      })
14    }
15  }
16)

To follow back a follower we can use .post('friendships/create'... here the bot is following back the user MarcGuberti

A bot should only follow users that follow the bot.

1bot.post(
2  'friendships/create',
3  {
4    screen_name: 'MarcGuberti',
5  },
6  (err, data, response) => {
7    if (err) {
8      console.log(err)
9    } else {
10      console.log(data)
11    }
12  }
13)

Like with followers you can get a list of accounts that your bot is following back.

1bot.get(
2  'friends/ids',
3  {
4    screen_name: 'DroidScott',
5  },
6  (err, data, response) => {
7    if (err) {
8      console.log(err)
9    } else {
10      console.log(data)
11    }
12  }
13)

And also a detailed list.

1bot.get(
2  'friends/list',
3  {
4    screen_name: 'DroidScott',
5  },
6  (err, data, response) => {
7    if (err) {
8      console.log(err)
9    } else {
10      console.log(data)
11    }
12  }
13)

Get friendship status, this is useful for following new followers, this will give us the relation of a specific user. So you can run through your followers list and follow back any users that do not have the following connection.

Let's take a look at the relation between our bot and @spences10

1bot.get(
2  'friendships/lookup',
3  {
4    screen_name: 'spences10',
5  },
6  (err, data, response) => {
7    if (err) {
8      console.log(err)
9    } else {
10      console.log(data)
11    }
12  }
13)

If the user follows the bot, then relationship will be:

1[ { name: 'Scott Spence 🌯😴💻♻',
2    screen_name: 'spences10',
3    id: 4897735439,
4    id_str: '4897735439',
5    connections: [ 'followed_by' ] } ]

If the user and the bot are following each other, the relationship will be:

1[ { name: 'Scott Spence 🌯😴💻♻',
2    screen_name: 'spences10',
3    id: 4897735439,
4    id_str: '4897735439',
5    connections: [ 'following', 'followed_by' ] } ]

And if there is no relationship then:

1[ { name: 'Scott Spence 🌯😴💻♻',
2    screen_name: 'spences10',
3    id: 4897735439,
4    id_str: '4897735439',
5    connections: [ 'none' ] } ]

Direct Message a user with bot.post('direct_messages/new'...

A bot should only DM a user that is following the bot account

1bot.post(
2  'direct_messages/new',
3  {
4    screen_name: 'spences10',
5    text: 'Hello from bot!',
6  },
7  (err, data, response) => {
8    if (err) {
9      console.log(err)
10    } else {
11      console.log(data)
12    }
13  }
14)

Interact with tweets

To get a list of tweets in the bots time line use .get(statuses/home_timeline'...

1bot.get(
2  'statuses/home_timeline',
3  {
4    count: 1,
5  },
6  (err, data, response) => {
7    if (err) {
8      console.log(err)
9    } else {
10      console.log(data)
11    }
12  }
13)

To be more granular you can pull out specific information on each tweet.

1bot.get(
2  'statuses/home_timeline',
3  {
4    count: 5,
5  },
6  (err, data, response) => {
7    if (err) {
8      console.log(err)
9    } else {
10      data.forEach(t => {
11        console.log(t.text)
12        console.log(t.user.screen_name)
13        console.log(t.id_str)
14        console.log('\n')
15      })
16    }
17  }
18)

To retweet use .post('statuses/retweet/:id'... and pass in a tweet id to retweet.

1bot.post(
2  'statuses/retweet/:id',
3  {
4    id: '860828247944253440',
5  },
6  (err, data, response) => {
7    if (err) {
8      console.log(err)
9    } else {
10      console.log(`${data.text} retweet success!`)
11    }
12  }
13)

To unretweet just use .post('statuses/unretweet/:id'...

1bot.post(
2  'statuses/unretweet/:id',
3  {
4    id: '860828247944253440',
5  },
6  (err, data, response) => {
7    if (err) {
8      console.log(err)
9    } else {
10      console.log(`${data.text} unretweet success!`)
11    }
12  }
13)

To like a tweet use .post('favorites/create'...

1bot.post(
2  'favorites/create',
3  {
4    id: '860897020726435840',
5  },
6  (err, data, response) => {
7    if (err) {
8      console.log(err)
9    } else {
10      console.log(`${data.text} tweet liked!`)
11    }
12  }
13)

To unlike a post use .post('favorites/destroy'...

1bot.post(
2  'favorites/destroy',
3  {
4    id: '860897020726435840',
5  },
6  (err, data, response) => {
7    if (err) {
8      console.log(err)
9    } else {
10      console.log(`${data.text} tweet unliked!`)
11    }
12  }
13)

To reply to a tweet is much the same a posting a tweet but you need to include the in_reply_to_status_id parameter, but that's not enough as you will also need to put in the screen name of the person you are replying to.

1bot.post(
2  'statuses/update',
3  {
4    status: '@spences10 I reply to you yes!',
5    in_reply_to_status_id: '860900406381211649',
6  },
7  (err, data, response) => {
8    if (err) {
9      console.log(err)
10    } else {
11      console.log(`${data.text} tweeted!`)
12    }
13  }
14)

Finally if you want to delete a tweet use .post('statuses/destroy/:id'... passing the tweet id you want to delete.

1bot.post(
2  'statuses/destroy/:id',
3  {
4    id: '860900437993676801',
5  },
6  (err, data, response) => {
7    if (err) {
8      console.log(err)
9    } else {
10      console.log(`${data.text} tweet deleted!`)
11    }
12  }
13)

Use Twitter search

To use search use .get('search/tweets',... there are quite a few search parameters for search.

q: '' the Q is for query so to search for mango use q: 'mango' we can also limit the results returned with count: n so let's limit it the count to in the example:

1bot.get(
2  'search/tweets',
3  {
4    q: 'mango',
5    count: 5,
6  },
7  (err, data, response) => {
8    if (err) {
9      console.log(err)
10    } else {
11      console.log(data.statuses)
12    }
13  }
14)

Like we did with the timeline we will pull out specific items from the data.statuses returned, like this:

1bot.get(
2  'search/tweets',
3  {
4    q: 'mango',
5    count: 5,
6  },
7  (err, data, response) => {
8    if (err) {
9      console.log(err)
10    } else {
11      data.statuses.forEach(s => {
12        console.log(s.text)
13        console.log(s.user.screen_name)
14        console.log('\n')
15      })
16    }
17  }
18)

The search API returns for relevance and not completeness, if you want to search for an exact phrase you'll need to wrap the query in quotes "purple pancakes" if you want to search for one of two words then use OR like 'tabs OR spaces' if you want to search for both use AND like 'tabs AND spaces'.

If you want to search for a tweet without another word use - like donald -trump you can use it multiple times as well, like donald -trump -duck

You can search for tweets with emoticons, like q: 'sad :(' try it!

Of course look for hashtags q: '#towie'. Look for tweets to a user q: 'to:@stephenfry' or from a user q: 'from:@stephenfry'

You can filter out indecent tweets with the filter:safe parameter you can also use it to filter for media tweets which will return tweets containing video. You can specify for images to view tweets with images and you can specify links for tweets with links.

If you want tweets from a certain website you can specify with the url parameter like url:asda

1bot.get(
2  'search/tweets',
3  {
4    q:
5      'from:@dan_abramov url:facebook filter:images since:2017-01-01',
6    count: 5,
7  },
8  (err, data, response) => {
9    if (err) {
10      console.log(err)
11    } else {
12      data.statuses.forEach(s => {
13        console.log(s.text)
14        console.log(s.user.screen_name)
15        console.log('\n')
16      })
17    }
18  }
19)

Last few now, there's the result_type parameter that will return recent, popular or mixed results.

The geocode parameter that take the format latitude longitude then radius in miles '51.5033640,-0.1276250,1mi' example:

1bot.get(
2  'search/tweets',
3  {
4    q: 'bacon',
5    geocode: '51.5033640,-0.1276250,1mi',
6    count: 5,
7  },
8  (err, data, response) => {
9    if (err) {
10      console.log(err)
11    } else {
12      data.statuses.forEach(s => {
13        console.log(s.text)
14        console.log(s.user.screen_name)
15        console.log('\n')
16      })
17    }
18  }
19)

Use Twitter Stream API

There are two ways to use the Stream API first there's .stream('statuses/sample') example:

1const stream = bot.stream('statuses/sample')
2
3stream.on('tweet', t => {
4  console.log(`${t.text}\n`)
5})

This will give you a random sampling of tweets.

For more specific information use .stream('statuses/filter')... then pass some parameters, use track: to specify a search string:

1var stream = bot.stream('statuses/filter', {
2  track: 'bot',
3})
4
5stream.on('tweet', function (t) {
6  console.log(t.text + '\n')
7})

You can also use multiple words in the track parameter, tis will get you results with either twitter or bot in them.

1const stream = bot.stream('statuses/filter', {
2  track: 'twitter, bot',
3})
4
5stream.on('tweet', t => {
6  console.log(`${t.text}\n`)
7})

If you want both words then remove the comma , you can think of spaces as AND and commas as OR

You can also use the follow: parameter which lets you input the ids of specific users, example:

1const stream = bot.stream('statuses/filter', {
2  follow: '4897735439',
3})
4
5stream.on('tweet', t => {
6  console.log(`${t.text}\n`)
7})

Tweet media files

This egghead.io video is a great resource for this section thanks to Hannah Davis for the awesome content!

This will be a request to get the NASA image of the day and tweet it.

For this we will need references to request and fs for working with the file system.

1const Twit = require('twit')
2const request = require('request')
3const fs = require('fs')
4const config = require('./config')
5
6const bot = new Twit(config)

First up get the photo from the NASA api, for this we will need to create a parameter object inside our getPhoto function that will be passed to the node HTTP client request for the image:

1function getPhoto() {
2  const parameters = {
3    url: 'https://api.nasa.gov/planetary/apod',
4    qs: {
5      api_key: process.env.NASA_KEY,
6    },
7    encoding: 'binary',
8  }
9}

The parameters specify an api_key for this you can apply for an API key or you can use the DEMO_KEY this API key can be used for initially exploring APIs prior to signing up, but it has much lower rate limits, so you’re encouraged to signup for your own API key.

In the example you can see that I have configured my key with the rest of my .env variables.

1CONSUMER_KEY=AmMSbxxxxxxxxxxNh4BcdMhxg
2CONSUMER_SECRET=eQUfMrHbtlxxxxxxxxxxkFNNj1H107xxxxxxxxxx6CZH0fjymV
3ACCESS_TOKEN=7xxxxx492-uEcacdl7HJxxxxxxxxxxecKpi90bFhdsGG2N7iII
4ACCESS_TOKEN_SECRET=77vGPTt20xxxxxxxxxxxZAU8wxxxxxxxxxx0PhOo43cGO
5
6NASA_KEY=DEMO_KEY

Now to use the request to get the image:

1function getPhoto() {
2  const parameters = {
3    url: 'https://api.nasa.gov/planetary/apod',
4    qs: {
5      api_key: process.env.NASA_KEY,
6    },
7    encoding: 'binary',
8  }
9  request.get(parameters, (err, respone, body) => {
10    body = JSON.parse(body)
11    saveFile(body, 'nasa.jpg')
12  })
13}

In the request we pass in our parameters and parse the body as JOSN so we can save it with the saveFile function which we'll go over now:

1function saveFile(body, fileName) {
2  const file = fs.createWriteStream(fileName)
3  request(body)
4    .pipe(file)
5    .on('close', err => {
6      if (err) {
7        console.log(err)
8      } else {
9        console.log('Media saved!')
10        console.log(body)
11      }
12    })
13}

request(body).pipe(file).on('close'... is what saves the file from the file variable which has the name passed to it nasa.jpg from the getPhoto function.

Calling getPhoto() should now save the NASA image of the day to the root of your project.

Now we can share it on Twitter 😎

Two parts to this, first save the file.

1function saveFile(body, fileName) {
2  const file = fs.createWriteStream(fileName)
3  request(body)
4    .pipe(file)
5    .on('close', err => {
6      if (err) {
7        console.log(err)
8      } else {
9        console.log('Media saved!')
10        const descriptionText = body.title
11        uploadMedia(descriptionText, fileName)
12      }
13    })
14}

Then uploadMedia to upload media to Twitter before we can post it, this had me stumped for a bit as I have my files in a src folder, if you have your bot files nested in folders then you will need to do the same if you are struggling with file does not exist errors:

Add a require to path then use join with the relevant relative file path.

1const path = require('path')
2//...
3const filePath = path.join(__dirname, '../' + fileName)

Complete function here:

1function uploadMedia(descriptionText, fileName) {
2  console.log(`uploadMedia: file PATH ${fileName}`)
3  bot.postMediaChunked(
4    {
5      file_path: fileName,
6    },
7    (err, data, respone) => {
8      if (err) {
9        console.log(err)
10      } else {
11        console.log(data)
12        const params = {
13          status: descriptionText,
14          media_ids: data.media_id_string,
15        }
16        postStatus(params)
17      }
18    }
19  )
20}

Then with the params we created in uploadMedia we can post with a straightforward .post('statuses/update'...

1function postStatus(params) {
2  bot.post('statuses/update', params, (err, data, respone) => {
3    if (err) {
4      console.log(err)
5    } else {
6      console.log('Status posted!')
7    }
8  })
9}

Call the getPhoto() function top post to Twitter... super straight forward, right 😀 no, I know it wasn't. Here's the complete module:

Click to expand

1const Twit = require('twit')
2const request = require('request')
3const fs = require('fs')
4const config = require('./config')
5const path = require('path')
6
7const bot = new Twit(config)
8
9function getPhoto() {
10  const parameters = {
11    url: 'https://api.nasa.gov/planetary/apod',
12    qs: {
13      api_key: process.env.NASA_KEY,
14    },
15    encoding: 'binary',
16  }
17  request.get(parameters, (err, respone, body) => {
18    body = JSON.parse(body)
19    saveFile(body, 'nasa.jpg')
20  })
21}
22
23function saveFile(body, fileName) {
24  const file = fs.createWriteStream(fileName)
25  request(body)
26    .pipe(file)
27    .on('close', err => {
28      if (err) {
29        console.log(err)
30      } else {
31        console.log('Media saved!')
32        const descriptionText = body.title
33        uploadMedia(descriptionText, fileName)
34      }
35    })
36}
37
38function uploadMedia(descriptionText, fileName) {
39  const filePath = path.join(__dirname, `../${fileName}`)
40  console.log(`file PATH ${filePath}`)
41  bot.postMediaChunked(
42    {
43      file_path: filePath,
44    },
45    (err, data, respone) => {
46      if (err) {
47        console.log(err)
48      } else {
49        console.log(data)
50        const params = {
51          status: descriptionText,
52          media_ids: data.media_id_string,
53        }
54        postStatus(params)
55      }
56    }
57  )
58}
59
60function postStatus(params) {
61  bot.post('statuses/update', params, (err, data, respone) => {
62    if (err) {
63      console.log(err)
64    } else {
65      console.log('Status posted!')
66    }
67  })
68}
69
70getPhoto()

Make a Markov bot

This is pretty neat, again from the egghead.io series it uses rita natural language toolkit. It also uses csv-parse as we're going to be reading out our Twitter archive to make the bot sound like us tweeting.

First of all, to set up the Twitter archive, you'll need to request your data from the Twitter settings page. You'll be emailed a link to download your archive, then when you have downloaded the archive extract out the tweets.csv file, we'll then put that in it's own folder, so from the root of your project:

1cd src
2mkdir twitter-archive

We'll move our tweets.csv there to be accessed by the bot we're going to go over now.

Use fs to set up a read stream...

1const filePath = path.join(__dirname, './twitter-archive/tweets.csv')
2
3const tweetData = fs
4  .createReadStream(filePath)
5  .pipe(
6    csvparse({
7      delimiter: ',',
8    })
9  )
10  .on('data', row => {
11    console.log(row[5])
12  })

When you run this from the console you should get the output from your Twitter archive.

Now clear out things like @ and RT to help with the natural language processing we'll set up two functions cleanText and hasNoStopWords

cleanText will tokenize the text delimiting it on space ' ' filter out the stop words then .join(' ') back together with a space and .trim() any whitespace that may be at the start of the text.

1function cleanText(text) {
2  return rita.RiTa.tokenize(text, ' ')
3    .filter(hasNoStopWords)
4    .join(' ')
5    .trim()
6}

The tokenized text can then be fed into the hasNoStopWords function to be sanitized for use in tweetData

1function hasNoStopWords(token) {
2  const stopwords = ['@', 'http', 'RT']
3  return stopwords.every(sw => !token.includes(sw))
4}

Now that we have the data cleaned we can tweet it, so replace console.log(row[5]) with inputText = inputText + ' ' + cleanText(row[5]) then we can use rita.RiMarkov(3) the 3 being the number of words to take into consideration. Then use markov.generateSentences(1) with 1 being the number of sentences being generated. We'll also use .toString() and .substring(0, 140) to truncate the result down to 140 characters.

1const tweetData =
2  fs.createReadStream(filePath)
3  .pipe(csvparse({
4    delimiter: ','
5  }))
6  .on('data', function (row) {
7    inputText = `${inputText} ${cleanText(row[5])}`
8  })
9  .on('end', function(){
10    const markov = new rita.RiMarkov(3)
11    markov.loadText(inputText)
12    const sentence = markov.generateSentences(1)
13      .toString()
14      .substring(0, 140)
15  }

Now we can tweet this with the bot using .post('statuses/update'... passing in the sentence variable as the status logging out when there is a tweet.

1const tweetData =
2  fs.createReadStream(filePath)
3    .pipe(csvparse({
4      delimiter: ','
5    }))
6    .on('data', row => {
7      inputText = `${inputText} ${cleanText(row[5])}`
8    })
9    .on('end', () => {
10      const markov = new rita.RiMarkov(3)
11      markov.loadText(inputText)
12      const sentence = markov.generateSentences(1)
13        .toString()
14        .substring(0, 140)
15      bot.post('statuses/update', {
16        status: sentence
17      }, (err, data, response) => {
18        if (err) {
19          console.log(err)
20        } else {
21          console.log('Markov status tweeted!', sentence)
22        }
23      })
24    })
25}

If you want your sentences to be closer to the input text you can increase the words to consider in rita.RiMarkov(6) and if you want to make it gibberish then lower the number.

Here's the completed module:

Click to expand

1const Twit = require('twit')
2const fs = require('fs')
3const csvparse = require('csv-parse')
4const rita = require('rita')
5const config = require('./config')
6const path = require('path')
7
8let inputText = ''
9
10const bot = new Twit(config)
11
12const filePath = path.join(__dirname, '../twitter-archive/tweets.csv')
13
14const tweetData =
15  fs.createReadStream(filePath)
16    .pipe(csvparse({
17      delimiter: ','
18    }))
19    .on('data', row => {
20      inputText = `${inputText} ${cleanText(row[5])}`
21    })
22    .on('end', () => {
23      const markov = new rita.RiMarkov(10)
24      markov.loadText(inputText)
25      const sentence = markov.generateSentences(1)
26        .toString()
27        .substring(0, 140)
28      bot.post('statuses/update', {
29        status: sentence
30      }, (err, data, response) => {
31        if (err) {
32          console.log(err)
33        } else {
34          console.log('Markov status tweeted!', sentence)
35        }
36      })
37    })
38}
39
40function hasNoStopWords(token) {
41  const stopwords = ['@', 'http', 'RT']
42  return stopwords.every(sw => !token.includes(sw))
43}
44
45function cleanText(text) {
46  return rita.RiTa.tokenize(text, ' ')
47    .filter(hasNoStopWords)
48    .join(' ')
49    .trim()
50}

Retrieve and Tweet data from Google sheets

If you want to tweet a list of links you can use tabletop to work though the list, in this example again from egghead.io we'll go through a list of links.

So, set up the bot and require tabletop:

1const Twit = require('twit')
2const config = require('./config')
3const Tabletop = require('tabletop')
4
5const bot = new Twit(config)

On your Google spreadsheet you'll need to have a header defined and then add your links, we'll use the following for an example:

links
https://www.freecodecamp.com
https://github.com
https://www.reddit.com
https://twitter.com

Now from Google sheets we can select 'File'>'Publish to the web' and copy the link that is generated we can use that in table top.

Now init Table top with three parameters, key: which is the spreadsheet URL, a callback: function to get the data and simpleSheet: which is true if you only have one sheet, like in our example here:

1const spreadsheetUrl =
2  'https://docs.google.com/spreadsheets/d/1842GC9JS9qDWHc-9leZoEn9Q_-jcPUcuDvIqd_MMPZQ/pubhtml'
3
4Tabletop.init({
5  key: spreadsheetUrl,
6  callback(data, tabletop) {
7    console.log(data)
8  },
9  simpleSheet: true,
10})

Running the bot now should give output like this:

1$ node index.js
2[ { 'links': 'https://www.freecodecamp.com' },
3  { 'links': 'https://github.com' },
4  { 'links': 'https://www.reddit.com' },
5  { 'links': 'https://twitter.com' } ]

So now we can tweet them using .post('statuses/update',... with a forEach on the data that is returned in the callback:

1Tabletop.init({
2  key: spreadsheetUrl,
3  callback(data, tabletop) {
4    data.forEach(d => {
5      const status = `${d.links} a link from a Google spreadsheet`
6      bot.post(
7        'statuses/update',
8        {
9          status,
10        },
11        (err, response, data) => {
12          if (err) {
13            console.log(err)
14          } else {
15            console.log('Post success!')
16          }
17        }
18      )
19    })
20  },
21  simpleSheet: true,
22})

Note that ${d.links} is the header name we use in the Google spreadsheet, I tried using skeleton and camel case and both returned errors so I went with a single name header on the spreadsheet.

The completed code here:

Click to expand

1const Twit = require('twit')
2const config = require('./config')
3const Tabletop = require('tabletop')
4
5const bot = new Twit(config)
6
7const spreadsheetUrl =
8  'https://docs.google.com/spreadsheets/d/1842GC9JS9qDWHc-9leZoEn9Q_-jcPUcuDvIqd_MMPZQ/pubhtml'
9
10Tabletop.init({
11  key: spreadsheetUrl,
12  callback(data, tabletop) {
13    data.forEach(d => {
14      const status = `${d.links} a link from a Google spreadsheet`
15      console.log(status)
16      bot.post(
17        'statuses/update',
18        {
19          status,
20        },
21        (err, response, data) => {
22          if (err) {
23            console.log(err)
24          } else {
25            console.log('Post success!')
26          }
27        }
28      )
29    })
30  },
31  simpleSheet: true,
32})

Putting it all together

Ok, so those examples were good n' all but we haven't really got a bot out of this have we? I mean you run it from the terminal and that's it done, we want to be able to kick off the bot and leave it to do its thing.

One way I have found to do this is to use setInterval which will kick off events from the main bot.js module, so let's try this:

Take the example we did to tweet a picture and add it to it's own module, so from the root directory of our project:

1cd src
2touch picture-bot.js

Take the example code from that and paste it into the new module, then we're going to make the following changes, to getPhoto:

1const getPhoto = () => {
2  const parameters = {
3    url: 'https://api.nasa.gov/planetary/apod',
4    qs: {
5      api_key: process.env.NASA_KEY,
6    },
7    encoding: 'binary',
8  }
9  request.get(parameters, (err, respone, body) => {
10    body = JSON.parse(body)
11    saveFile(body, 'nasa.jpg')
12  })
13}

Then at the bottom of the module add:

1module.exports = getPhoto

So now we can call the getPhoto function from the picture-bot.js module in our bot.js module, our bot.js module should look something like this:

1const picture = require('./picture-bot')
2
3picture()

That's it, two lines of code, try running that from the terminal now:

1yarn start

We should get some output like this:

1yarn start v0.23.4
2$ node index.js
3Media saved!
4file PATH C:\Users\path\to\project\tweebot-play\nasa.jpg
5{ media_id: 863020197799764000,
6  media_id_string: '863020197799763968',
7  size: 371664,
8  expires_after_secs: 86400,
9  image: { image_type: 'image/jpeg', w: 954, h: 944 } }
10Status posted!
11Done in 9.89s.

Ok, so thats the picture of the day done, but it has run once and completed we need to put it on an interval with setInterval which we need to pass two options to, the function it's going to call and the timeout value.

The picture updates every 24 hours so that will be how many milliseconds in 24 hours [8.64e+7] I don't even 🤷‍

I work it out like this, 1000 60 = 1 minute, so 1000 60 60 24 so for now let's add that directly into the setInterval function:

1const picture = require('./picture-bot')
2
3picture()
4setInterval(picture, 1000 * 60 * 60 * 24)

Cool, that's a bot that will post the NASA image of the day every 24 hours!

Let's keep going, now let's add some randomness in with the Markov bot, like we did in the picture of the day example, let's create a new module for the Markov bot and add all the code in there from the previous example, so from the terminal:

1cd src
2touch markov-bot.js

Then copy pasta the markov bot example into the new module, then we're going to make the following changes:

1const tweetData = () => {
2  fs.createReadStream(filePath)
3    .pipe(
4      csvparse({
5        delimiter: ',',
6      })
7    )
8    .on('data', row => {
9      inputText = `${inputText} ${cleanText(row[5])}`
10    })
11    .on('end', () => {
12      const markov = new rita.RiMarkov(10)
13      markov.loadText(inputText).toString().substring(0, 140)
14      const sentence = markov.generateSentences(1)
15      bot.post(
16        'statuses/update',
17        {
18          status: sentence,
19        },
20        (err, data, response) => {
21          if (err) {
22            console.log(err)
23          } else {
24            console.log('Markov status tweeted!', sentence)
25          }
26        }
27      )
28    })
29}

Then at the bottom of the module add:

1module.exports = tweetData

Ok, same again as with the picture bot example we're going to add the tweetData export from markov-bot.js to our bot.js module, which should now look something like this:

1const picture = require('./picture-bot')
2const markov = require('./markov-bot')
3
4picture()
5setInterval(picture, 1000 * 60 * 60 * 24)
6
7markov()

Let's make the Markov bot tweet at random intervals between 5 minutes and 3 hours

1const picture = require('./picture-bot')
2const markov = require('./markov-bot')
3
4picture()
5setInterval(picture, 1000 * 60 * 60 * 24)
6
7const markovInterval = (Math.floor(Math.random() * 180) + 1) * 1000
8markov()
9setInterval(markov, markovInterval)

Allrighty! Picture bot, Markov bot, done 👍

Do the same with the link bot? Ok, same as before, you get the idea now, right?

Create a new file in the src folder for link bot:

1touch link-bot.js

Copy pasta the code from the link bot example into the new module, like this:

1const link = () => {
2  Tabletop.init({
3    key: spreadsheetUrl,
4    callback(data, tabletop) {
5      data.forEach(d => {
6        const status = `${d.links} a link from a Google spreadsheet`
7        console.log(status)
8        bot.post(
9          'statuses/update',
10          {
11            status,
12          },
13          (err, response, data) => {
14            if (err) {
15              console.log(err)
16            } else {
17              console.log('Post success!')
18            }
19          }
20        )
21      })
22    },
23    simpleSheet: true,
24  })
25}
26
27module.exports = link

Then we can call it from the bot, so it should look something like this:

1const picture = require('./picture-bot')
2const markov = require('./markov-bot')
3const link = require('./link-bot')
4
5picture()
6setInterval(picture, 1000 * 60 * 60 * 24)
7
8const markovInterval = (Math.floor(Math.random() * 180) + 1) * 1000
9markov()
10setInterval(markov, markovInterval)
11
12link()
13setInterval(link, 1000 * 60 * 60 * 24)

Ok? Cool 👍😎

We can now leave the bot running to do its thing!!

Deploy to now

Right, we have a bot that does a few things but it's on our development environment, so it can't stay there forever, well it could but it'd be pretty impcratcical. Let's put our bot on a server somewhere to do it's thing.

To do this we're going to be using now, now allows for simple deployments from the CLI if you're not fimailiar with now then take a quick look at the documentation in these examples we're going to be using the now-cli.

There's a few things we need to do in order to get our bot ready to go on now, let's list them quickly and then go into detail.

Signup and install now-cli
Add now settings + .npmignore file
Add .env variables as secrets
Add npm deploy script
Re jig picture-bot.js

Ready? Let's do this! 💪

Signup and install now-cli

Fist up let's signup for zeit ▲ create an account and authenticate, then we can install the CLI.

Install now globally on our machine so you can use it everywhere, to install the now-cli from the terminal enter:

1npm install -g now

Once it's completed login with:

1now --login

The first time you run now, it'll ask for your email address in order to identify you. Go to the email account to supplied when sigining up an click on the email sent to you from now, and you'll be logged in automatically.

If you need to switch the account or re-authenticate, run the same command again.

You can always check out the now-cli documentation for more information along with the your first deployment guide.

Add now settings

Ok, so that's signup and install sorted, we can now configure the bot for deploying to now. First up let's add the now settings to our package.json file, I've put it in between my npm scripts and the author name in my package.json:

1"scripts": {
2    "start": "node index.js"
3  },
4  "now": {
5    "alias": "my-awesome-alias",
6    "files": [
7      "src",
8      "index.js"
9    ]
10  },
11  "author": "Scott Spence",

This was a source of major confusion for me so I'm hoping I can save you the pain I went through trying to configure this, all the relevant documentation is there you just need to put it all together 😎

If you find anything in here that doesn't make sense or is just outright wrong then please log an issue or create a pull request 👍

The now settings alias is to give your deployment a shothand name over the auto generated URL that now creates, the files section covers what we want to include in the depoloyment to now we'll go over the file structure shortly. Basically what is included in the files array is all that get pused up to the now servers.

All good so for?

Ok, now we need to add a .npmignore file in the root of the project and add the following line to it:

1!tweets.csv

The tweets.csv needs to go up to the now server to be used by the bot, but we previously included it in our .gitignore which is what now uses to build your project when it's being loaded to the server. So this means that the file isn't going to get loaded unless we add the .npmignore to not ignore the tweets.csv 😅

Add .env variables as secrets

Ok, our super duper secret Twitter keys will need to be stored as secrets in now this is a pretty neat feature where you can define anything as a secret and reference it as an alias with now.

Let's start, so the syntax is now secrets add my-secret "my value" so for our .env keys add them all in giving them a descriptive [but short!] name.

You will not need to wrap your "my value" in quotes but the documentation does say "when in doubt, wrap your value in quotes"

Ok, so from the terminal now secrets ls should list out your secrets you just created:

1$ now secrets ls
2> 5 secrets found under spences10 [1s]
3
4                            id  name                   created
5  sec_xxxxxxxxxxZpLDxxxxxxxxxx  ds-twit-key            23h ago
6  sec_xxxxxxxxxxTE5Kxxxxxxxxxx  ds-twit-secret         23h ago
7  sec_xxxxxxxxxxNorlxxxxxxxxxx  ds-twit-access         23h ago
8  sec_xxxxxxxxxxMe1Cxxxxxxxxxx  ds-twit-access-secret  23h ago
9  sec_xxxxxxxxxxMJ2jxxxxxxxxxx  nasa-key               23h ago

Add npm deploy script

Now we have our secrets defined we can create a deployment script to deploy to now, so in our package.json let's add an additional script:

1"main": "index.js",
2  "scripts": {
3    "start": "node index.js",
4    "deploy": "now -e CONSUMER_KEY=@ds-twit-key
5        -e CONSUMER_SECRET=@ds-twit-secret -e ACCESS_TOKEN=@ds-twit-access
6        -e ACCESS_TOKEN_SECRET=@ds-twit-access-secret -e NASA_KEY=@nasa-key"
7  },
8  "now": {

Let's go over what we have added there, deploy will run the now command and pass it all our environment -e variables and the associated secret value, if we break it down into separate lines it will be a bit clearer:

1now
2-e CONSUMER_KEY=@ds-twit-key
3-e CONSUMER_SECRET=@ds-twit-secret
4-e ACCESS_TOKEN=@ds-twit-access
5-e ACCESS_TOKEN_SECRET=@ds-twit-access-secret
6-e NASA_KEY=@nasa-key

Re jig picture-bot.js

Ok, because now deployments are immutable it means that there's no write access to the disk where we want to save our NASA photo of the day, so to get around that we need to use the /tmp file location.

Shout out to @Tim from zeit for helping me out with this!

In the picture-bot.js module add the following two lines to the top of the module:

1const os = require('os')
2const tmpDir = os.tmpdir()

Those two lines give us the temp directory of the operating system, so if like me you're on Windows it will work as well as if you are on another stsyem like a linux based system, which is what now is. In our saveFile function we're going to use tmpDir to save our file.

We've taken out the nasa.jpg from the getPhoto function as we can define that information in the saveFile function, the NASA potd is not just a 'jpeg some items posted there are videos as well. We we can define the type with a ternary function off of the body being passed in, this will send a tweet with a link to the video:

1function saveFile(body) {
2  const fileName =
3    body.media_type === 'image/jpeg' ? 'nasa.jpg' : 'nasa.mp4'
4  const filePath = path.join(tmpDir + `/${fileName}`)
5
6  console.log(`saveFile: file PATH ${filePath}`)
7  if (fileName === 'nasa.mp4') {
8    // tweet the link
9    const params = {
10      status: 'NASA video link: ' + body.url,
11    }
12    postStatus(params)
13    return
14  }
15  const file = fs.createWriteStream(filePath)
16
17  request(body)
18    .pipe(file)
19    .on('close', err => {
20      if (err) {
21        console.log(err)
22      } else {
23        console.log('Media saved!')
24        const descriptionText = body.title
25        uploadMedia(descriptionText, filePath)
26      }
27    })
28}

The completed code here:

Click to expand

1const Twit = require('twit')
2const request = require('request')
3const fs = require('fs')
4const config = require('./config')
5const path = require('path')
6
7const bot = new Twit(config)
8
9const os = require('os')
10const tmpDir = os.tmpdir()
11
12const getPhoto = () => {
13  const parameters = {
14    url: 'https://api.nasa.gov/planetary/apod',
15    qs: {
16      api_key: process.env.NASA_KEY,
17    },
18    encoding: 'binary',
19  }
20  request.get(parameters, (err, respone, body) => {
21    body = JSON.parse(body)
22    saveFile(body)
23  })
24}
25
26function saveFile(body) {
27  const fileName =
28    body.media_type === 'image/jpeg' ? 'nasa.jpg' : 'nasa.mp4'
29  const filePath = path.join(tmpDir + `/${fileName}`)
30
31  console.log(`saveFile: file PATH ${filePath}`)
32  if (fileName === 'nasa.mp4') {
33    // tweet the link
34    const params = {
35      status: 'NASA video link: ' + body.url,
36    }
37    postStatus(params)
38    return
39  }
40  const file = fs.createWriteStream(filePath)
41
42  request(body)
43    .pipe(file)
44    .on('close', err => {
45      if (err) {
46        console.log(err)
47      } else {
48        console.log('Media saved!')
49        const descriptionText = body.title
50        uploadMedia(descriptionText, filePath)
51      }
52    })
53}
54
55function uploadMedia(descriptionText, fileName) {
56  console.log(`uploadMedia: file PATH ${fileName}`)
57  bot.postMediaChunked(
58    {
59      file_path: fileName,
60    },
61    (err, data, respone) => {
62      if (err) {
63        console.log(err)
64      } else {
65        console.log(data)
66        const params = {
67          status: descriptionText,
68          media_ids: data.media_id_string,
69        }
70        postStatus(params)
71      }
72    }
73  )
74}
75
76function postStatus(params) {
77  bot.post('statuses/update', params, (err, data, respone) => {
78    if (err) {
79      console.log(err)
80    } else {
81      console.log('Status posted!')
82    }
83  })
84}
85
86module.exports = getPhoto

Ok, thats it! We're ready to deploy to now!🚀

So from the terminal we call our deployment script we defined earlier:

1yarn deploy

You will get some output:

1λ yarn deploy
2yarn deploy v0.24.4
3$ now -e CONSUMER_KEY=@ds-twit-key -e CONSUMER_SECRET=@ds-twit-secret
4    -e ACCESS_TOKEN=@ds-twit-access  -e ACCESS_TOKEN_SECRET=@ds-twit-access-secret
5    -e NASA_KEY=@nasa-key
6> Deploying ~\gitrepos\tweebot-play under spences10
7> Using Node.js 7.10.0 (default)
8> Ready! https://twee-bot-play-rapjuiuddx.now.sh (copied to clipboard) [5s]
9> Upload [====================] 100% 0.0s
10> Sync complete (1.54kB) [2s]
11> Initializing…
12> Building
13> ▲ npm install
14> ⧗ Installing:
15>  ‣ csv-parse@^1.2.0
16>  ‣ dotenv@^4.0.0
17>  ‣ rita@^1.1.63
18>  ‣ tabletop@^1.5.2
19>  ‣ twit@^2.2.5
20> ✓ Installed 106 modules [3s]
21> ▲ npm start
22> > tweet-bot-playground@1.0.0 start /home/nowuser/src
23> > node index.js
24> saveFile: file PATH /tmp/nasa.jpg
25> Media saved!
26> uploadMedia: file PATH /tmp/nasa.jpg

Woot! You have your bot deployed! 🙌

If you click on the link produced you will be able to inspect the bot as it is on now there's also a handy logs section on the page where you can check for output. 👍

Contributing

Please fork this repository and contribute back using pull requests.

Any contributions, large or small, major features, bug fixes and integration tests are welcomed and appreciated but will be thoroughly reviewed and discussed.

License

MIT License

Twitter bot playground

Set up the bot

Post Statuses

Work with users

Interact with tweets

Use Twitter search

Use Twitter Stream API

Tweet media files

Make a Markov bot

Retrieve and Tweet data from Google sheets

Putting it all together

Deploy to now

Contributing

License

Useful? Click here to share this post on Twitter.