How I Built a Twitter Keyword Monitoring Using a Serverless Node.js Function With AWS Amplify
In this article, I will demonstrate to you how I built a simple serverless Node.js function on AWS that sends me a daily email with a list of tweets that mention me on Twitter.
Recently, I used Twilert and Birdspotter for that purpose, which are specialized tools for Twitter keyword monitoring. But their free plans/trials don't fulfill my simple requirements, so I decided to implement them independently.
Prerequisites
I chose again AWS Amplify to deploy the serverless function to AWS.
If you don't already have an AWS account, you'll need to create one to follow the steps outlined in this article. Please follow this tutorial to create an account.
Next, you need to install and configure the Amplify Command Line Interface (CLI).
The serverless function will need access to secrets stored in the AWS Secret Manager. My article “How to Use Environment Variables to Store Secrets in AWS Amplify Backend” will guide you through this process.
Add Serverless Function to AWS
The first step is to add a new Lambda (serverless) function with the Node.js runtime to the Amplify application.
The function gets invoked on a recurring schedule. In my case, it will be invoked every day at 08:00 PM.
Let's add the serverless function using the Amplify CLI:
1▶ amplify add function
2? Select which capability you want to add: Lambda function (serverless function)
3? Provide an AWS Lambda function name: twittersearchfunction
4? Choose the runtime that you want to use: NodeJS
5? Choose the function template that you want to use: Hello World
6? Do you want to configure advanced settings? Yes
7? Do you want to access other resources in this project from your Lambda function? No
8? Do you want to invoke this function on a recurring schedule? Yes
9? At which interval should the function be invoked: Daily
10? Select the start time (use arrow keys): 08:00 PM
11? Do you want to enable Lambda layers for this function? No
12? Do you want to configure environment variables for this function? No
13? Do you want to configure secret values this function can access? No
14? Do you want to edit the local lambda function now? No
Get a list of tweets for a specific Twitter keyword
Now it's time to write the JavaScript code that returns a list of tweets for a given keyword.
Let's start by writing the twitter-client.js
module. This module uses FeedHive’s Twitter Client to access the Twitter API. The first step is to initialize the Twitter API client and trigger the request:
1const mokkappsTwitterId = 481186762
2const searchQuery = 'mokkapps'
3const searchResultCount = 100
4
5const fetchRecentTweets = async (secretValues) => {
6 // Configure Twitter API Client
7 const twitterClient = new twitterApiClient.TwitterClient({
8 apiKey: secretValues.TWITTER_API_KEY,
9 apiSecret: secretValues.TWITTER_API_KEY_SECRET,
10 accessToken: secretValues.TWITTER_ACCESS_TOKEN,
11 accessTokenSecret: secretValues.TWITTER_ACCESS_TOKEN_SECRET,
12 })
13
14 // Trigger search endpoint: https://github.com/FeedHive/twitter-api-client/blob/main/REFERENCES.md#twitterclienttweetssearchparameters
15 const searchResponse = await twitterClient.tweets.search({
16 q: searchQuery,
17 count: searchResultCount,
18 result_type: 'recent',
19 })
20
21 // Access statuses from response
22 const statuses = searchResponse.statuses
23}
Next, we want to filter the response into three groups:
- Tweets: Tweets from the last 24 hours that were not published by my Twitter account and are no replies or retweets
- Replies: Tweets from the last 24 hours that were not published by my Twitter account and are replies
- Retweets: Tweets from the last 24 hours that were not published by my Twitter account and are retweets
Let's start by the filtering the statuses
response for "normal" tweets that are no replies or retweets:
1const isTweetedInLast24Hours = (status) => {
2 const tweetDate = new Date(status.created_at)
3 const now = new Date()
4 const timeDifference = now.getTime() - tweetDate.getTime()
5 const daysDifference = timeDifference / (1000 * 60 * 60 * 24)
6 return daysDifference <= 1
7}
8
9const fetchRecentTweets = async (secretValues) => {
10 // ...
11 const statuses = searchResponse.statuses
12
13 const tweets = statuses.filter((status) => {
14 const isNotOwnAccount = status.user.id !== mokkappsTwitterId
15 const isNoReply = status.in_reply_to_status_id === null
16 const isNoRetweet = status.retweeted_status === null
17 return isNotOwnAccount && isNoReply && isNoRetweet && isTweetedInLast24Hours(status)
18 })
19}
Now we can filter for retweets and replies in a similar way:
1const retweets = statuses.filter((status) => {
2 const isNotOwnAccount = status.user.id !== mokkappsTwitterId
3 const isRetweet = status.retweeted_status
4 return isNotOwnAccount && isRetweet && isTweetedInLast24Hours(status)
5})
6
7const replies = statuses.filter((status) => {
8 const isNotOwnAccount = status.user.id !== mokkappsTwitterId
9 const isReply = status.in_reply_to_status_id !== null
10 return isNotOwnAccount && isReply && isTweetedInLast24Hours(status)
11})
The last step is to map the results to a very simple HTML structure that will be rendered inside the email body:
1const { formatDistance } = require('date-fns')
2
3const mapStatus = (status) => {
4 const {
5 id_str: id,
6 created_at,
7 in_reply_to_screen_name,
8 in_reply_to_status_id_str,
9 text,
10 retweet_count,
11 favorite_count,
12 user: { screen_name: user_screen_name, followers_count, created_at: userCreatedAt, friends_count },
13 } = status
14 const createdAtLocaleString = new Date(created_at).toLocaleString()
15 const url = `https://twitter.com/${user_screen_name}/status/${id}`
16 const userUrl = `https://twitter.com/${user_screen_name}`
17 const originalUrl = in_reply_to_screen_name
18 ? `https://twitter.com/${in_reply_to_screen_name}/status/${in_reply_to_status_id_str}`
19 : null
20 const userCreatedDateDistance = formatDistance(new Date(), new Date(userCreatedAt))
21
22 return `
23 <div style="margin-bottom: 20px; padding: 10px; border: 1px solid gray; border-radius: 5px;">
24 <h2>From <a href=${userUrl}>${user_screen_name}</a> at ${createdAtLocaleString}</h2>
25 <small><strong>Followers:</strong> ${followers_count}, <strong>Following:</strong> ${friends_count}, <strong>Account Created:</strong> ${userCreatedDateDistance} ago</small>
26 <h3>${text}</h3>
27 <a href=${url} style="margin-top: 10px">Tweet</a>
28 <small style="margin-top: 5px">(<strong>Likes:</strong> ${favorite_count}, <strong>Retweets: ${retweet_count})</strong></small>
29 ${originalUrl ? `<div style="margin-top: 10px"></br><a href=${originalUrl}>Original Tweet</a></div>` : ''}
30 </div>
31 `
32}
33
34const fetchRecentTweets = async (secretValues) => {
35 // ...
36 const retweets = statuses
37 .filter((status) => {
38 const isNotOwnAccount = status.user.id !== mokkappsTwitterId
39 const isRetweet = status.retweeted_status
40 return isNotOwnAccount && isRetweet && isTweetedInLast24Hours(status)
41 })
42 .map((status) => mapStatus(status))
43}
This is the code for the whole twitter-client.js
module:
1const twitterApiClient = require('twitter-api-client')
2const { formatDistance } = require('date-fns')
3
4const mokkappsTwitterId = 481186762
5const searchQuery = 'mokkapps'
6const searchResultCount = 100
7
8const mapStatus = (status) => {
9 const {
10 id_str: id,
11 created_at,
12 in_reply_to_screen_name,
13 in_reply_to_status_id_str,
14 text,
15 retweet_count,
16 favorite_count,
17 user: { screen_name: user_screen_name, followers_count, created_at: userCreatedAt, friends_count },
18 } = status
19 const createdAtLocaleString = new Date(created_at).toLocaleString()
20 const url = `https://twitter.com/${user_screen_name}/status/${id}`
21 const userUrl = `https://twitter.com/${user_screen_name}`
22 const originalUrl = in_reply_to_screen_name
23 ? `https://twitter.com/${in_reply_to_screen_name}/status/${in_reply_to_status_id_str}`
24 : null
25 const userCreatedDateDistance = formatDistance(new Date(), new Date(userCreatedAt))
26
27 return `
28 <div style="margin-bottom: 20px; padding: 10px; border: 1px solid gray; border-radius: 5px;">
29 <h2>From <a href=${userUrl}>${user_screen_name}</a> at ${createdAtLocaleString}</h2>
30 <small><strong>Followers:</strong> ${followers_count}, <strong>Following:</strong> ${friends_count}, <strong>Account Created:</strong> ${userCreatedDateDistance} ago</small>
31 <h3>${text}</h3>
32 <a href=${url} style="margin-top: 10px">Tweet</a>
33 <small style="margin-top: 5px">(<strong>Likes:</strong> ${favorite_count}, <strong>Retweets: ${retweet_count})</strong></small>
34 ${originalUrl ? `<div style="margin-top: 10px"></br><a href=${originalUrl}>Original Tweet</a></div>` : ''}
35 </div>
36 `
37}
38
39const isTweetedInLast24Hours = (status) => {
40 const tweetDate = new Date(status.created_at)
41 const now = new Date()
42 const timeDifference = now.getTime() - tweetDate.getTime()
43 const daysDifference = timeDifference / (1000 * 60 * 60 * 24)
44 return daysDifference <= 1
45}
46
47const fetchRecentTweets = async (secretValues) => {
48 const twitterClient = new twitterApiClient.TwitterClient({
49 apiKey: secretValues.TWITTER_API_KEY,
50 apiSecret: secretValues.TWITTER_API_KEY_SECRET,
51 accessToken: secretValues.TWITTER_ACCESS_TOKEN,
52 accessTokenSecret: secretValues.TWITTER_ACCESS_TOKEN_SECRET,
53 })
54
55 const searchResponse = await twitterClient.tweets.search({
56 q: searchQuery,
57 count: searchResultCount,
58 result_type: 'recent',
59 })
60
61 const statuses = searchResponse.statuses
62
63 const tweets = statuses
64 .filter((status) => {
65 const isNotOwnAccount = status.user.id !== mokkappsTwitterId
66 const isNoReply = status.in_reply_to_status_id === null
67 const isNoRetweet = status.retweeted_status === null
68 return isNotOwnAccount && isNoReply && isNoRetweet && isTweetedInLast24Hours(status)
69 })
70 .map((status) => mapStatus(status))
71
72 const retweets = statuses
73 .filter((status) => {
74 const isNotOwnAccount = status.user.id !== mokkappsTwitterId
75 const isRetweet = status.retweeted_status
76 return isNotOwnAccount && isRetweet && isTweetedInLast24Hours(status)
77 })
78 .map((status) => mapStatus(status))
79
80 const replies = statuses
81 .filter((status) => {
82 const isNotOwnAccount = status.user.id !== mokkappsTwitterId
83 const isReply = status.in_reply_to_status_id !== null
84 return isNotOwnAccount && isReply && isTweetedInLast24Hours(status)
85 })
86 .map((status) => mapStatus(status))
87
88 return {
89 tweets,
90 retweets,
91 replies,
92 }
93}
94
95module.exports = fetchRecentTweets
Serverless function code
We can now use the twitter-client.js
in our serverless function:
1const AWS = require('aws-sdk')
2const nodemailer = require('nodemailer')
3const fetchRecentTweets = require('./twitter-client')
4
5const secretsManager = new AWS.SecretsManager()
6const responseHeaders = {
7 'Content-Type': 'application/json',
8}
9
10exports.handler = async (event) => {
11 console.log(`👷 Function is ready to search for tweets`)
12
13 const secretData = await secretsManager.getSecretValue({ SecretId: 'YOUR_SECRET_ID' }).promise()
14 const secretValues = JSON.parse(secretData.SecretString)
15
16 const transporter = nodemailer.createTransport({
17 service: secretValues.MAIL_HOST,
18 auth: {
19 user: secretValues.MAIL_USER,
20 pass: secretValues.MAIL_PW,
21 },
22 })
23
24 const defaultMailOptions = {
25 from: secretValues.MAIL_USER,
26 to: secretValues.MAIL_SUCCESS,
27 subject: `[Mokkapps API] Twitter Search Results`,
28 }
29
30 try {
31 // Fetch recent tweets
32 const { tweets, replies, retweets } = await fetchRecentTweets(secretValues)
33
34 // Skip sending email if we have no results
35 if (tweets.length === 0 && replies.length === 0 && retweets.length === 0) {
36 return {
37 statusCode: 200,
38 headers: responseHeaders,
39 body: [],
40 }
41 }
42
43 // Send email
44 await transporter.sendMail({
45 ...defaultMailOptions,
46 html: `
47 <h1>Tweets that mentioned "mokkapps" in the last 24 hours</h1>
48 ${tweets.length === 0 ? '<p>No results</p>' : tweets.join('')}
49 <h1>Replies that mentioned "mokkapps" in the last 24 hours</h1>
50 ${replies.length === 0 ? '<p>No results</p>' : replies.join('')}
51 <h1>Retweets that mentioned "mokkapps" in the last 24 hours</h1>
52 ${retweets.length === 0 ? '<p>No results</p>' : retweets.join('')}
53 `,
54 })
55
56 return {
57 statusCode: 200,
58 headers: responseHeaders,
59 body: JSON.stringify({ tweets, replies, retweets }),
60 }
61 } catch (e) {
62 console.error('☠ Twitter Search Function Error:', e)
63 return {
64 statusCode: 500,
65 headers: responseHeaders,
66 body: e.message ? e.message : JSON.stringify(e),
67 }
68 }
69}
At this point, we can publish our function by running:
amplify push
If we successfully pushed the function to AWS, we can manually invoke the function in AWS Lamba by clicking the "Test" button:
The serverless function should then send an email with a list of tweets if someone mentioned the monitored keyword in the last 24 hours:
Conclusion
I had a lot of fun building this simple serverless function to monitor keywords on Twitter.
Serverless functions are a perfect choice for such a monitoring tool, as we only have to pay for the execution time of the serverless function.
What do you think about my solution? Leave a comment and tell me how you monitor your Twitter keywords.
If you liked this article, follow me on Twitter to get notified about new blog posts and more content from me.
Alternatively (or additionally), you can also subscribe to my newsletter.
Document & Test Vue 3 Components With Storybook
Storybook is my tool of choice for UI component documentation. Vue.js is very well supported in the Storybook ecosystem and has first-class integrations with Vuetify and NuxtJS. It also has official support for Vue 3, the latest major installment of Vue.js.
The 10 Favorite Features of My Developer Portfolio Website
Inspired by Braydon Coyer's new blogfolio, I've added some excellent new features to my portfolio website.