Multimedia

Cloud Phone is designed for rich multimedia experiences, thanks to powerful server-side rendering. Many popular apps like YouTube and TikTok are available today.

Images

Cloud Phone processes HTML, CSS, and JavaScript into standard Puffin Vector Graphics Language (PVGL) to overcome limitations in native device rendering. For more refresher, learn about Cloud Phone architecture.

Supported Image Formats

• Bitmap (BMP)
• Portable Network Graphics (PNG)
• Joint Photographic Experts Group (JPEG)
• Scalable Vector Graphics (SVG)
• Graphics Interchange Format (GIF)
• AV1 Image File Format (AVIF)
• WebP

Responsive Images

Cloud Phone automatically downsamples (resizes & compresses) images to fit small screens and limited device random access memory (RAM), often as low as 16MB! Cloud Phone supports responsive images, including both the srcset attribute and <picture> element. Responsive images can save bandwidth, reduce cost, improve page load times, and ensure images are displayed correctly.

Unsupported Image Formats

Cloud Phone does not support image formats that are not packaged with Chromium by default. Unsupported formats include JPEG XL, High Efficiency Image File Format (HEIF/HEIC), Tag Image File Format (TIFF), Truevision Graphics Adapter (TGA), Silicon Graphics Image (SGI), PhotoShop Document (PSD), and other proprietary or RAW image formats.

Audio

Cloud Phone supports streaming remote audio over HTTPS, and local audio via Blob URLs.

Supported Audio Formats

Container Formats

• MP4
• Ogg
• WebM
• WAV

Codec Formats

• FLAC
• MP3
• Opus
• Vorbis
• AAC

WebAudio

The Web Audio API is not supported at this time.

Volume Manager API

W3C Audio/ Video Standards define the volume property to control playback volume between 0.0 and 1.0 but, “the loudest setting may be lower than the system’s loudest possible setting.” This is to prevent disruptive and awkward experiences where visiting a website unexpectedly causes a loud advertisement to play.

However, many Cloud Phone devices lack a physical volume rocker. This makes it difficult to raise and lower the volume without exiting an application.

To overcome these limitations, Cloud Phone provides the Volume Manager API to raise and lower the system volume. These APIs display the volume heads up display (HUD), which differs between devices in both style and number of increments.

The Volume Manager API lets users select an appropriate volume without leaving your widget. The API consists of three functions that all return void. While the HUD is visible, widgets cannot intercept the ArrowUp or ArrowDown keys as these raise and lower the volume, respectively.

// Displays the Volume HUD
// While visible:
// - ArrowUp raises volume
// - ArrowDown lowers volume
navigator.volumeManager.requestShow();

// Displays the Volume HUD and RAISES volume
navigator.volumeManager.requestUp();

// Displays the Volume HUD and LOWERS volume
navigator.volumeManager.requestDown();

HTMLMediaElement.volume does nothing on Cloud Phone. Use navigator.volumeManager instead.

interface VolumeManager {
  requestShow(): void;
  requestUp(): void;
  requestDown(): void;
}

declare global {
  interface Navigator {
    volumeManager?: VolumeManager;
  }
}

Autoplay

The autoplay policy for Cloud Phone is the same as Google Chrome.

• Muted autoplay is always allowed
• Autoplay with sound is allowed after the user has interacted with your widget
• Autoplay is enabled on frames based on the Permissions Policy set by the allow attribute

The autoplay attribute can be set on <audio> and <video> elements. Alternatively, begin with muted autoplay and let users unmute using a keydown event.

If permitted, calling play() on a media element will return a Promise that resolves, or rejects with an error if autoplay is not allowed.

mediaElement.play()
  .then(() => {
    // Autoplay started!
  })
  .catch((error) => {
    // Autoplay prevented
    // Display a "Play" button for user to click
  });

If your Cloud Phone widget shares a codebase with another platform, there are libraries like can-autoplay designed to detect when autoplay is enabled. The experimental Navigator.getAutoplayPolicy() and non-standard allowedToPlay attributes are not supported at this time.

Playback Rate

HTMLMediaElement.playbackRate does not work on Cloud Phone. Audio and video are always at 1.0 “normal speed.”

Video

Codecs

Cloud Phone supports the MPEG-4 (MP4) video file format, including the H.264 standard with AAC audio. Cloud Phone may support other codecs supported native by Google Chrome, but we recommend H.264 + AAC for an optimal streaming experience.

Inline Playback

Use the custom x-puffin-playsinline attribute to disable Cloud Phone’s default fullscreen <video> element display. This is particularly useful for short, small videos that might have otherwise been rendered as inline GIFs.

<video x-puffin-playsinline>

HTTP Live Streaming (HLS)

Cloud Phone natively supports adaptive bitrate streaming without libraries like HLS.js. Behind the scene, Cloud Phone is compatible with ffmpeg 5.1 including support for most of HLSv7.

<video width="240" height="320" controls>
  <source src="playlist.m3u8" type="application/x-mpegURL" />
</video>

Cloud Phone automatically selects the lowest bitrate stream to approximate the resolution that most closely matches QQVGA or QVGA screens. This may select audio-only streams if those are included and lower bitrate than video streams. In the example M3U8 playlist file, the third audio-only stream has the lowest BANDWIDTH of 57544, so it would be selected over the two video streams.

#EXTM3U
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=506651,RESOLUTION=640x480
https://example.com/stream?id=0
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=155108,RESOLUTION=320x240
https://example.com/stream?id=1
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=57544,CODECS="mp4a.40.2"
https://example.com/stream?id=2

Feature Detection

You can feature detect native HLS support. This is especially useful if your sharing a codebase with another platform.

function hasNativeHLS(): boolean {
  return videoEl.canPlayType("application/vnd.apple.mpegurl") === "probably";
}

Seeking

Video seeking has been supported since Cloud Phone 2.5, and can be detected using the Feature Detection API with the VideoSeek feature name.

function canVideoSeek(): Promise<boolean> {
  return navigator.hasFeature('VideoSeek');
}

Unsupported APIs

Media Source Extensions API

Cloud Phone does not support the Media Source Extension API. Check for the presence of the MediaSource interface and fall back to other media sources as needed.

function hasMediaSource(): boolean {
  return window.MediaSource !== undefined;
}

Encrypted Media Extension API

Cloud Phone does not support the Encrypted Media Extension API needed to play content restricted by Digital Rights Management (DRM). Although APIs like navigator.requestMediaKeySystemAccess are available on Cloud Phone, the following key systems are unavailable at this time:

• Widevine: com.widevine.alpha
• PlayReady: com.microsoft.playready, com.youtube.playready
• ClearKey: webkit-org.w3.clearkey, org.w3.clearkey
• PrimeTime: com.adobe.primetime, com.adobe.access
• FairPlay: com.apple.fairplay

WebRTC

WebRTC (Web Real-Time Communication) APIs for candidate selection and peer-to-peer connections are not supported on Cloud Phone at this time.

You can connect to the microphone using getUserMedia() on Cloud Phone 3.0 or higher, and detect microphone support using the AudioCapture feature.

Summary

Cloud Phone has extensive support for streaming audio and video, including native support for live-streaming. With the same codec and container format as Chrome, you don’t need to make changes on Cloud Phone to accommodate the limited hardware of low-end feature phones.