Person, Flooring, Wood, Floor, Hardwood, Furniture, Plywood, Couch, Sphere

Managing Videos on Android

Grey Skold | Android Video Product

Video launched on the Pinterest Android app in 2016 with the goal of bringing a seamless video experience to the app. This included the ability to support multiple videos per screen, while dynamically controlling their playback state by automatically pausing them when they’ve scrolled offscreen, or controlling the number of videos allowed to play simultaneously.

We soon discovered there were many technical challenges we needed to address, such as:

  • Managing the playback state of all currently available videos
  • Knowing a video’s on-screen visibility percentage
  • Providing our developers an easy-to-use video component

Over time we’ve gradually reworked our video architecture to tackle these requirements, and below we’ll dive into how we handled these challenges in our latest video module.

The Video Manager

At a high level, we needed to build a component that would be aware of all video instances (i.e. Views) as well as their related surfaces (i.e. Fragments) available on-screen. Managing the surfaces is critical for monitoring the lifecycle states (i.e. onStart() etc.) that will be applied to the surface’s children, and avoids adding excessive code on the consumer-layer to apply the latest state changes to the Views.

For tracking these key lifecycle events, the Android framework provides us the current state of both the content being shown on-screen, as well as any changes visually affecting our app. The key lifecycle events we listen for are the UI attachment calls (e.g. onAttachedToWindow()) as well as when our host screen changes its display state (e.g. onPause() etc.).

With these callback methods, we attempted to register any videos that have been provided a valid video URL. This will give us our initial list of videos available within our current surface.

In the first iteration of our video framework, we relied on the client code invoking these calls themselves, but as we found this to be unscalable, as it added more complexity when building video features. Instead, we abstracted away the callbacks for registering a video behind the VideoManager by building methods that required the underlying video component to be passed in. From there, the VideoManager would make appropriate calculations behind the scenes. This eliminated the need for consumers to already have predefined knowledge of the video registration process, as it now just worked “out-of-the-box”.

Before

// FooBarFragment.class for FooBar feature
override fun onResume() {
     super.onResume()
     // Required by consumers to implement
     videoView?.apply {
          viewability = Viewability.FullyVisible
          onActivate()
          onViewCompletelyVisible()
     }
}
override fun onPause() {
     // Required by consumers to implement
     videoView?.apply {
          viewability = Viewability.NotVisible
          onDeactivate()
     }
     super.onPause()
}

After

// BaseFragment.class implemented by all screens
override fun onResume() {
     super.onResume()
     videoManager.onResume(this)
}
override fun onPause() {
     videoManager.onPause(this)
     super.onPause()
}
// VideoManager.class internally
override fun onResume(videoSurface: VideoViewSurface) {
     videoSurface.videoViews.forEach {
          registerVideo(it)
     }
}
override fun onPause(videoSurface: VideoViewSurface) {
     videoSurface.videoViews.forEach {
          unregisterVideo(it)
     }
}

Holding onto this list of videos gave us the ability to dynamically set the playback state based on the current visibility of our app. This also provided the flexibility to dynamically change other functionality based on certain metadata properties passed in upon video registration.

For example, we may want all video ads to autoplay, but limit to having only 1 organic video (i.e. Creator generated content) autoplaying on the same surface. By checking the metadata registered on an individual video, we can apply these limitations onto the UI-layer.

We also abstracted away all Pinterest-specific analytics code in order to maintain a clear focus for the Video Manager (managing and playing videos), and to keep the component app-agnostic.

Calculating Viewability

Viewability is defined as the percentage of visible area of a UI component shown on-screen. This measurement is crucial in our understanding of what is currently being displayed to the user. With this information we are able to gather information for our partners regarding the engagement of their content.

In the common case, since the VideoManager holds references to all active videos, we can track the exact coordinate of our views (i.e. getLocationInWindow()) and our device’s screen size in pixels (see DisplayMetrics) to deduce their Viewability on-screen.

We also handle overlapping UI via:

  • Providing consumers the option to include a list of “obstruction” Views that could potentially draw over our underlying videos (e.g. toolbars, floating buttons, etc.)
  • Callbacks for pop-ups being displayed (i.e. onWindowFocusChanged())
  • Components for screen scrolling or UI components going off-screen (see RecyclerView listeners)
  • Additional callbacks for when a video surface is shown on-screen (i.e. onResume() etc.).

Building for Developers

While we wanted to reduce the amount of video management complexity exposed to our developers, the largest area of confusion was implementing a new video surface. Therefore,we both abstracted away the complexity for video setup, as well as utilized UI components provided by Google’s PlayerView:

Before

// FooBar video feature, requires custom FooBarVideoView.class of 100+ lines
object : FooBarVideoView(
     context,     // application context
     analytics,   // Analytics object
     url,         // video url
     uid,         // unique ID
     false        // isAd flag
) {
     // configuration flag for custom setup (mute, autoplay, controller, etc.)
     override val videoConfiguration = VideoConfiguration.FOO_BAR
}.apply {
     shouldLoop = true
     videoAspectRatio = aspectRatio
     render(videoMetaData) // loads video, videoMetaData contains: url, isAd, uid
}

After

// Foobar video feature, no custom class required just set flags
PinterestVideoView(context).apply {
     // Optional params for setup/customization
     this.analytics = pinterestAnalytics
     this.mute = false
     this.autoPlay = true
     this.alwaysAutoplay = true
     this.alwaysPlay = true
     this.showMute = true
     this.looping = true
     this.bufferingRule = SHOW_BUFFERING_ALWAYS
}.apply {
     render(videoMetaData) // loads video, videoMetaData contains: url, isAd, uid
}

Another complexity of the video infrastructure was the actual VideoManager architecture itself. In our rewrite, we consolidated a majority of our old components to only the bare bones needed to support a functioning VideoManager

Before

After

Our new VideoManager architecture provides a clear hierarchy of events and components’ relation to one another. This not only looks good on paper, but the refactor alone has removed over ~4,500 lines of code (less than 1/3 the size of the original implementation).

Playing off into the sunset…

Building proper “video management” is a long and arduous process, but over the years we’ve built something that’s truly evolved to help both streamline our development process and Pinner experience. In the future, we hope to open source our work so other developers may contribute to the ongoing effort to handle dynamic video playback. We will continue to iterate on our video client architecture tackling new challenges as they come, with the goal of building a delightful video experience for both Pinners and developers.

We’re building the world’s first visual discovery engine. More than 250 million people around the world use Pinterest to dream about, plan and prepare for things they want to do in life. Come join us!

Managing Videos on Android was originally published in Pinterest Engineering Blog on Medium, where people are continuing the conversation by highlighting and responding to this story.

Read more >

Jobs people like you viewed

Trust & Safety Technical Strategist

San Francisco California United States San Francisco, California, United States Policy Community Operations > Policy
We’re looking for a curious problem-solver to join our Trust & Safety team as a Technical Specialist. With your top-notch critical thinking skills, you’ll define metrics and build dashboards to monitor the performance of our Trust & Safety...

SWE, Ads Reporting Infra

San Francisco California United States San Francisco, California, United States Engineering Engineering > Eng
Pinterest is one of the fastest growing online advertising platforms, and our continued success depends on our ability to generate reliable, accurate, real-time data to advertisers, business analysts, and Ads Serving systems, and empower these int...

Director of Agency & Industry Marketing

London England United Kingdom London, England, United Kingdom Marketing Brand, Marketing and Communications > Partner Marketing
We’re looking for a Director of Agency and Industry Marketing to bring inspiration and thought leadership to the agency and marketing communities in North America and Europe. You'll lead a team of marketers based in New York and Europe in developi...

Business Recruiter (Contract)

San Francisco California United States San Francisco, California, United States Recruiting People > Recruiting
Pinterest brings millions of people the inspiration to create a life they love for everything; whether that be tonight’s dinner, next summer’s vacation, or a dream house down the road. We’re looking for a Business Recruiter to help attract and hir...

Product Counsel

San Francisco California United States San Francisco, California, United States Legal and Policy Legal and Policy > Legal
Pinterest brings millions of people the inspiration to create a life they love. We’re looking for a practical lawyer to join our legal team to support the expansion of our ads product and policy teams, with a focus on our global monetization produ...

Intellectual Property Operations Specialist

San Francisco California United States San Francisco, California, United States Community Operations Community Operations > Pinner Operations
Pinterest brings millions of Pinners the inspiration to create a life they love for everything; whether that be tonight’s dinner, next summer’s vacation, or a dream house down the road. We’re looking for a motivated self-starter to join our Intell...