Google Summer of Code Ideas

Table of contents

Information for students

To learn more about GPAC, you can find information about the project history, some metrics, and a short description.

As indicated on KDE’s GSoC ideas page:

If you wish to submit a proposal based on these ideas, you may wish to contact the developers and find out more about the particular suggestion you’re looking at this page.

Being accepted as a Google Summer of Code student is quite competitive. Accepted students typically have thoroughly researched the technologies of their proposed project and have been in frequent contact with potential mentors. Simply copying and pasting an idea here will not work. On the other hand, creating a completely new idea without first consulting potential mentors is unlikely to work out.

If you have any question, please contact gsoc@gpac.io.

Ideas

Summary of ideas.

Idea 1 : MP4Box.js, the Web MP4 swiss-army knife

Description

MP4, and more generally the ISOBMFF, has become an unavoidable format for media on the Web. The GPAC project offers an excellent support for MP4 file manipulations and playback using its standalone command-line tools: MP4Box or MP4Client. However, in many situations, MP4 manipulations should be done directly in browsers.

MP4Box.js is the latest tool of the GPAC project. Its goal is to help creators and users of MP4 files to process those files in browsers, or more generally in JavaScript based environments like Node JS. The project has already many features such as: file inspection, on-the-fly fragmentation, processing in service workers, …

Expected results

The goals of this project is to improve MP4Box.js in the following directions:

  • Testing: the project currently uses QUnit for basic testing, but more advanced testing is required, to check browser differences for instance or to check playback of large files automatically.
  • Performance: the project currently uses different bit/byte manipulation libraries. The idea is to benchmark the different libraries, come up with a recommendation and modify the code to use the recommended approach.
  • Fragmentation: Current fragmentation of MP4 files produces a fragment per frame. While simple, it is not always efficient and produces latency in some browsers. The project should improve MP4 fragmentation.
  • Integration with JS media decoders: MP4 files can store many types of media data, but not everyting is supported by the native decoders of browsers. The design of an API from MP4Box.js to integrate with JS decoders such as a JS HEVC decoder could be interesting.
  • Service Workers: recent browsers support the use of the Streams API in Service Workers. This API could be used to improve MP4Box.js Service Worker.
  • Player release: the current demo of MP4Box.js-based player still misses many features of existing players. The project could help the reuse of the player.
  • Support for the community: users already as for many new features such as import of MKV files.

Required skills/knowledge

Proficiency in JavaScript coding is required and a good knowledge of multimedia formats and streaming technologies is desired.

Mentor

Cyril Concolato (cyril.concolato@gpac.io)
Backup Mentor: Romain Bouqueau (romain.bouqueau@gpac.io)

Idea 2 : Security and exploits on the GPAC codebase

Description

Security is important when software is exposed to attackers. The GPAC team builds software used by millions of people through demuxers and players. These tools include bitstream parsers and a network stacks which offer surface for attacks. GPAC needs a security policy.

Security tools are also more widely available to developers such as static analyzers (Clang scan build, Coverity as a service), compile tools and options (ASAN) or dynamic (Valgrind, DynamoRIO framework) or independent frameworks (Frama-C). These tools need to be integrated in the existing build and test infrastructure.

Security is also a question of good practices. The GPAC community also identified a few weak points that would highly improved the security experience and mitigate some hard-to-track crashes.

Expected results

Set up a security infrastructure and demonstrate improvements on objective metrics over the GSoC duration (e.g. Coverity).

Required skills/knowledge

Proficiency in C, including pointers and memory management.
Interest in security.

Mentor:

Romain Bouqueau (romain.bouqueau@gpac.io)
Backup Mentor: Jean Le Feuvre (jean.lefeuvre@gpac.io)

Idea 4 : Add support for new import formats to MP4Box (MKV/WebM, VP9)

Description

MP4Box is GPAC’s command line multiplexer for ISOBMFF (a.k.a. MP4). It already supports various input formats such as MPEG-2 Transport Stream and Program Stream, AVI files, raw AVC|H.264 and HEVC, etc…

The goal of this project is to add support for Matroska (MKV and WebM) files as input to the multiplexer, as well as adding support for VP8/VP9/VP10/Opus and other common codecs found in such files, in order to simplify conversion from mkv to mp4 and generation of MPEG-DASH content.

Documentation

Where to look at in the GPAC code?

Similar importers. For example look at AVI:

Matrovska:

WebM/VPX:

Expected results

Support for remultiplexing MKV/WebM files to MP4 and MPEG-DASH generation of the resulting files.

Required skills/knowledge

Cross-platform development (Linux/Windows/OSX)
Proficiency in C.
Basic multimedia knowledge. We can teach the trainee about the audio and video codecs and containers if needed.

Mentor:

Jean Le Feuvre (jean.lefeuvre@gpac.io)
Backup Mentor: Cyril Concolato (cyril.concolato@gpac.io)

Idea 5 : Add playback support for YouTube 360 videos

Description

Google have added a documented XML metadata to describe the orientation and projection of their 360 videos.

GPAC has native support for 360 videos from equirectilinear projections. This includes support for navigation.

The aim is to include Google metadata in GPAC. Some extensions would be to add support for Youtube URLs in GPAC or support more projections and formats from other popular 360 video providers (Facebook…).

Expected results

Play 360 videos from Youtube or other platforms with different projections.

Required skills/knowledge

Basic multimedia knowledge.
We can teach the trainee about the ISOBMF/MP4 muxer specifics.

Mentor:

Romain Bouqueau (romain.bouqueau@gpac.io)
Backup Mentor: Jean Le Feuvre (jean.lefeuvre@gpac.io)

Idea 6 : DashCastX: port the popular DashCast to the Signals platform

Description

The GPAC team has started a C++11 initiative called Signals. Signals is a modular framework. It allows to easily separate components, to provide clean testable APIs, and to connect these components together. Signals is simple for both module and application makers. The Signals core is complex but small, TDDed, and architectured with independent layers.

Prior to Signals the GPAC team had developed the DashCast tool to stream some live DASH streams. DashCast is a popular tool but is difficult to maintain.

The aim is to write a replacement for DashCast written with Signals.

Expected results

Flexible MPEG-DASH encoder with support for common audio and video codecs in multiple qualities, langages and/or codecs, replicating DashCast functionality for both off-line sources and live sources (webcams, screen cast).

Required skills/knowledge

C++11. Basic multimedia knowledge.

Mentor:

Romain Bouqueau (romain.bouqueau@gpac.io)
Backup Mentor: Jean Le Feuvre (jean.lefeuvre@gpac.io)

Idea 7 : Improved GPAC dependency build using Zenbuild

Description

The GPAC team has extended a build system called Zenbuild. Zenbuild aims at building projects which usually causes troubles (e.g. FFmpeg with librtmp). It could also help with some license issues when distributing the software is not possible (e.g. FFmpeg non-free).

The GPAC framework includes a player. The player has a lot of dependencies including those generated by Zenbuild. The detection and test of these dependencies is hard and generate a lot of support from the GPAC team.

The aim is to develop a script that would generate all the player dependencies using Zenbuild and other packages already available. This may require some tweaking in Zenbuild itself.

Expected results

Build GPAC dependencies on popular platforms (covered by our build infrastructure and TravisCI) from a single script. Allow a GPAC static build with zero dependency.

Required skills/knowledge

Basic build skills (make, shell script ; compile, link).
Knowledge on building on different platforms is a plus.

Mentor:

Romain Bouqueau (romain.bouqueau@gpac.io)
Backup Mentor: Rodolphe Fouquet (rodolphe.fouquet@gpac.io)