Skip to main content

Today I Learned

Small learnings documented in public

  • Mastodon User Timelines are RSS Feeds

    Topic: social media, mastodon, today i learnedTechnology:

    I was reading through the Mastodon API documentation to figure out how to get my posts.

    On this website, I have a small widget that shows my last "Tweets" but unfortunately I locked myself out of my Twitter account after accidentally setting my age to under 13, while deleting personal information. This means my Twitter timeline isn't available, so I made the jump and figured out how to use the Mastodon API to pull a user's timeline.

    For Twitter this is an entire thing:

    def auth():
        """ Get the Twitter Bearer for Authentification """
        return os.getenv('TWITTER_BEARER')
    
    def create_headers(bearer_token):
        """ Create Header for API Request """
        headers = {"Authorization": "Bearer {}".format(bearer_token)}
        return headers
    
    def create_url(user = 'JesperDramsch', max_results = 200):
        """ Create URL Schema """
        search_url = f"https://api.twitter.com/1.1/statuses/user_timeline.json?screen_name={user}&count={max_results}&exclude_replies=true&include_rts=false&trim_user=true" #Change to the endpoint you want to collect data from
    
        #change params based on the endpoint you are using
        query_params = {'next_token': {}}
        return (search_url, query_params)
    
    
    def connect_to_endpoint(url, headers, params, next_token = None):
        """ Get JSON from Twitter Endpoint """
        params['next_token'] = next_token   #params object received from create_url function
        response = requests.request("GET", url, headers = headers, params = params)
        print("Endpoint Response Code: " + str(response.status_code))
        if response.status_code != 200:
            raise Exception(response.status_code, response.text)
        return response.json()
    
    def process_tweets(json_response, max_results=5):
        """ Process JSON from Twitter response"""
        tweets = []
        num_tweets = 0
        for tweet in json_response:
            if "IFTTT" in tweet['source']:
                continue
            text = tweet['text'].replace("\n\n", "<br/>").replace("\n", "<br/>").replace("https://t.co", "<br/>https://t.co")
            created = datetime.strptime(tweet['created_at'],'%a %b %d %H:%M:%S +0000 %Y')
            date_format = '%B %d, %Y'
            tweetid = tweet['id_str']
    
            active = ' active' if not num_tweets else ''
            flag = False
            tweets.append(f'<div class="item{active}"><div class="tweet"><p>{text}</p></div><!--/.item --><div class="separator-container"><div class="separator"><div class="shape"></div></div></div><div class="time"><a href="https://twitter.com/JesperDramsch/status/{tweetid}"><time class="published dt-published" datetime="{created.isoformat()}" itemprop="datePublished" title="{created.strftime(date_format)}">{created.strftime(date_format)}</time></a> from <a href="https://twitter.com/JesperDramsch">@JesperDramsch</a></div></div>')
            num_tweets += 1
            if num_tweets == max_results:
                break
        return ''.join(tweets)
    
    bearer_token = auth()
    headers = create_headers(bearer_token)
    
    url = create_url()
    json_response = connect_to_endpoint(url[0], headers, url[1])
    
    last_tweets = process_tweets(json_response)
    print(last_tweets)
    

    So I figured I can reuse a bunch of this code and use the API endpoint for statuses or accounts...

    Absolutely unnecessary.

    Public Mastodon feeds are RSS feeds, so they're just as easy to parse as this blog's articles feed.

    def create_url(instance, user):
        """ Create URL Schema """
        search_url = f"https://{instance}/@{user}.rss"
        return search_url
    
    def connect_to_endpoint(url):
        """ Get Response from RSS Feed """"
        response = feedparser.parse(url).entries
        return response
    
    def process_tweets(json_response, max_results=5):
        """ Process JSON Response """
        tweets = []
        num_tweets = 0
        for tweet in json_response:
            text = tweet['summary_detail']['value']
            created = datetime.fromtimestamp(time.mktime(tweet['published_parsed']))
            date_format = '%B %d, %Y'
            tweetid = tweet['id']
    
            active = ' active' if not num_tweets else ''
            flag = False
            tweets.append(f'<div class="item{active}"><div class="tweet"><p>{text}</p></div><!--/.item --><div class="separator-container"><div class="separator"><div class="shape"></div></div></div><div class="time"><a href="{tweetid}"><time class="published dt-published" datetime="{created.isoformat()}" itemprop="datePublished" title="{created.strftime(date_format)}">{created.strftime(date_format)}</time></a> from <a href="/mastodon">@[email protected]</a></div></div>')
            num_tweets += 1
            if num_tweets == max_results:
                break
        return ''.join(tweets)
    
    instance = 'tech.lgbt'
    user = 'jesper'
    url = create_url(instance, user)
    json_response = connect_to_endpoint(url)
    
    last_tweets = process_tweets(json_response)
    print(last_tweets)
    

    The function process_tweets() is close to identical. But connecting is very easy. It's all RSS under the hood. TIL

    Here's mine for example: tech.lgbt/@jesper.rss

  • I learned about TIL posts

    Topic: content, today i learnedTechnology:

    Gotta love a bit of self-referential writing.

    I read about TIL (Today I Learned) posts by Simon Willison, as an easy way to document small snippets of learning and get in the habit of writing and publishing.

    Then I saw multiple people do this inspired by J Branchaud:

    https://github.com/jbranchaud/til

    I figured I'd add this section to my website to document my Learning in Public journey and keep writing.

    I learned a lot about Nikola in the process. There's no easy way to keep these short-form posts separate from the longer articles