Person, Flooring, Wood, Floor, Hardwood, Furniture, Plywood, Couch, Sphere

Privacy Power-Up: Upgrading Pin Links from HTTP to HTTPS

Maria de Angelis | Summer 2019 Pinterest Engineer Intern

HTTPS should be everywhere. Your website likely runs over HTTPS, giving your users a secure, encrypted connection, but what about all the outbound links? Can you guarantee that they all use HTTPS? In this post, we describe the steps we took to upgrade Pin links from HTTP to HTTPS by leveraging DuckDuckGo’s Smarter Encryption technology.

Why HTTPS?

While Pinterest runs entirely on HTTPS, it’s not the final destination for most users. As a place for Pinners to discover and do what they love, Pinterest is the launching pad for reaching other websites. As such, we have a responsibility to redirect Pinners to HTTPS sites whenever we can, including upgrading outbound Pin links from HTTP to HTTPS when possible and maximizing the percentage of traffic through HTTPS.

The reason that any HTTP traffic exists is simply that many Pins were created that way. When a Pinner clicks on a Pin link, she’ll be sent to the HTTP site if the content was created with an HTTP URL. However, if the site supports HTTPS, we want to send the user to the HTTPS version instead. We decided to perform an online upgrade to HTTPS so we could maximize the result on all Pinterest surfaces.

After making improvements, about 80% of outbound traffic is now through HTTPS, an increase of over 30%.

In order to ensure this change did not interfere with important metrics, we ran an experiment to release these changes to 1% of users and compare them to an equivalent control group. We found there was no change in benchmark Pinterest metrics, so we’ll continue to release the experiment to more Pinterest users.

Figure 1: This Pinterest board’s Pins are color coded by the protocol of their links. HTTPS links are green and HTTP links are red. When the experiment is enabled in the second picture, most red Pins become green because their links can be upgraded to HTTPS.

Smarter Encryption by DuckDuckGo

To work as efficiently as possible, we integrated DuckDuckGo’s Smarter Encryption technology which automatically uses encrypted connections to websites when available. DuckDuckGo was the perfect fit for us because they maintain a comprehensive list of upgradable sites, generated by comparing the HTTP and HTTPS version of a site, and adding a site to the HTTPS upgrade list if the two versions are identical. We can then regularly pull and ingest their list.

Pinterest Architecture

When a user scrolls through their Pinterest home feed, a request to fetch Pins is made in the API layer, which then calls the Apache Thrift service PinAndBoardService, which subsequently fetches Pins from the MySQL database.

Figure 2: A simplified representation of Pin fetching architecture. The API layer makes a request to the PinAndBoardService which then fetches the Pin from the cache or MySQL.

Overview

To implement Pin link conversion, we:

  • Trigger whether or not to run the experiment in the API layer.
  • If the experiment is enabled, call a newly added endpoint in PinAndBoardService in the API layer.
  • In the new PinAndBoardService endpoint, check if a Pin’s URL begins with HTTP when it is fetched from the database or cache, in which case it should be upgraded to HTTPS if possible.
  • Check if a Pin is able to be upgraded to HTTPS by stripping the domain from the URL and checking if the domain is contained in a list of secure domains provided by DuckDuckGo’s Smarter Encryption.
  • Perform the conversion and upgrade the link so that all downstream services will receive the secure version.
  • We chose to perform the link conversion in PinAndBoardService instead of directly in MySQL database for several reasons:
  • If we upgraded a HTTP link to HTTPS in the MySQL layer and later on that domain no longer supported HTTPS, the Pin link would break and we would not be able to track the change since the link would have been permanently modified. To solve this issue, we could have stored both the old and new links in MySQL, but that would have been less space efficient than the current approach.
  • Although we have to perform the conversion everytime a Pin gets fetched from the database (regardless of whether it has already been upgraded), we can generally avoid redundant fetching because Pins are highly cached.
  • Next steps

    As we move forward, we’re exploring permanently upgrading Pins to secure HTTPS links upon creation, a big step towards universal HTTPS usage and increased security for all users on our platform. We are also tracking the links that we cannot currently upgrade to potentially contribute to DuckDuckGo’s Smarter Encryption in order to improve the user experience for all users.

    Acknowledgements

    Huge thanks to Emanuele Cesena (my awesome intern mentor) and the rest of the Security team at Pinterest! Also huge thanks to the CoreService, Storage & Caching, and API teams for providing support and DuckDuckGo for providing early access to their Smarter Encryption technology.


    Privacy Power-Up: Upgrading Pin Links from HTTP to HTTPS was originally published in Pinterest Engineering Blog on Medium, where people are continuing the conversation by highlighting and responding to this story.

    Read more >

    Jobs people like you viewed

    Trust & Safety Technical Strategist

    San Francisco California United States San Francisco, California, United States Policy Community Operations > Policy
    We’re looking for a curious problem-solver to join our Trust & Safety team as a Technical Specialist. With your top-notch critical thinking skills, you’ll define metrics and build dashboards to monitor the performance of our Trust & Safety...

    SWE, Ads Reporting Infra

    San Francisco California United States San Francisco, California, United States Engineering Engineering > Eng
    Pinterest is one of the fastest growing online advertising platforms, and our continued success depends on our ability to generate reliable, accurate, real-time data to advertisers, business analysts, and Ads Serving systems, and empower these int...

    Director of Agency & Industry Marketing

    London England United Kingdom London, England, United Kingdom Marketing Brand, Marketing and Communications > Partner Marketing
    We’re looking for a Director of Agency and Industry Marketing to bring inspiration and thought leadership to the agency and marketing communities in North America and Europe. You'll lead a team of marketers based in New York and Europe in developi...

    Business Recruiter (Contract)

    San Francisco California United States San Francisco, California, United States Recruiting People > Recruiting
    Pinterest brings millions of people the inspiration to create a life they love for everything; whether that be tonight’s dinner, next summer’s vacation, or a dream house down the road. We’re looking for a Business Recruiter to help attract and hir...

    Product Counsel

    San Francisco California United States San Francisco, California, United States Legal and Policy Legal and Policy > Legal
    Pinterest brings millions of people the inspiration to create a life they love. We’re looking for a practical lawyer to join our legal team to support the expansion of our ads product and policy teams, with a focus on our global monetization produ...

    Intellectual Property Operations Specialist

    San Francisco California United States San Francisco, California, United States Community Operations Community Operations > Pinner Operations
    Pinterest brings millions of Pinners the inspiration to create a life they love for everything; whether that be tonight’s dinner, next summer’s vacation, or a dream house down the road. We’re looking for a motivated self-starter to join our Intell...