Table of contents
As indicated on KDE’s GSoC ideas page:
If you wish to submit a proposal based on these ideas, you may wish to contact the developers and find out more about the particular suggestion you’re looking at this page.
Being accepted as a Google Summer of Code student is quite competitive. Accepted students typically have thoroughly researched the technologies of their proposed project and have been in frequent contact with potential mentors. Simply copying and pasting an idea here will not work. On the other hand, creating a completely new idea without first consulting potential mentors is unlikely to work out.
If you have any question, please contact email@example.com.
Summary of ideas.
MP4, and more generally the ISOBMFF, has become an unavoidable format for media on the Web. The GPAC project offers an excellent support for MP4 file manipulations and playback using its standalone command-line tools: MP4Box or MP4Client. However, in many situations, MP4 manipulations should be done directly in browsers.
The goals of this project is to improve MP4Box.js in the following directions:
- Testing: the project currently uses QUnit for basic testing, but more advanced testing is required, to check browser differences for instance or to check playback of large files automatically.
- Performance: the project currently uses different bit/byte manipulation libraries. The idea is to benchmark the different libraries, come up with a recommendation and modify the code to use the recommended approach.
- Fragmentation: Current fragmentation of MP4 files produces a fragment per frame. While simple, it is not always efficient and produces latency in some browsers. The project should improve MP4 fragmentation.
- Integration with JS media decoders: MP4 files can store many types of media data, but not everyting is supported by the native decoders of browsers. The design of an API from MP4Box.js to integrate with JS decoders such as a JS HEVC decoder could be interesting.
- Service Workers: recent browsers support the use of the Streams API in Service Workers. This API could be used to improve MP4Box.js Service Worker.
- Player release: the current demo of MP4Box.js-based player still misses many features of existing players. The project could help the reuse of the player.
- Support for the community: users already as for many new features such as import of MKV files.
Security is important when software is exposed to attackers. The GPAC team builds software used by millions of people through demuxers and players. These tools include bitstream parsers and a network stacks which offer surface for attacks. GPAC needs a security policy.
Security tools are also more widely available to developers such as static analyzers (Clang scan build, Coverity as a service), compile tools and options (ASAN) or dynamic (Valgrind, DynamoRIO framework) or independent frameworks (Frama-C). These tools need to be integrated in the existing build and test infrastructure.
Security is also a question of good practices. The GPAC community also identified a few weak points that would highly improved the security experience and mitigate some hard-to-track crashes.
Set up a security infrastructure and demonstrate improvements on objective metrics over the GSoC duration (e.g. Coverity).
Proficiency in C, including pointers and memory management.
Interest in security.
MP4Box is GPAC’s command line multiplexer for ISOBMFF (a.k.a. MP4). It already supports various input formats such as MPEG-2 Transport Stream and Program Stream, AVI files, raw AVC|H.264 and HEVC, etc…
The goal of this project is to add support for Matroska (MKV and WebM) files as input to the multiplexer, as well as adding support for VP8/VP9/VP10/Opus and other common codecs found in such files, in order to simplify conversion from mkv to mp4 and generation of MPEG-DASH content.
Where to look at in the GPAC code?
Similar importers. For example look at AVI:
- Look for gf_import_avi*() functions.
Support for remultiplexing MKV/WebM files to MP4 and MPEG-DASH generation of the resulting files.
Cross-platform development (Linux/Windows/OSX)
Proficiency in C.
Basic multimedia knowledge. We can teach the trainee about the audio and video codecs and containers if needed.
GPAC has native support for 360 videos from equirectilinear projections. This includes support for navigation.
The aim is to include Google metadata in GPAC. Some extensions would be to add support for Youtube URLs in GPAC or support more projections and formats from other popular 360 video providers (Facebook…).
Play 360 videos from Youtube or other platforms with different projections.
Basic multimedia knowledge.
We can teach the trainee about the ISOBMF/MP4 muxer specifics.
The GPAC team has started a C++11 initiative called Signals. Signals is a modular framework. It allows to easily separate components, to provide clean testable APIs, and to connect these components together. Signals is simple for both module and application makers. The Signals core is complex but small, TDDed, and architectured with independent layers.
The aim is to write a replacement for DashCast written with Signals.
Flexible MPEG-DASH encoder with support for common audio and video codecs in multiple qualities, langages and/or codecs, replicating DashCast functionality for both off-line sources and live sources (webcams, screen cast).
C++11. Basic multimedia knowledge.
The GPAC team has extended a build system called Zenbuild. Zenbuild aims at building projects which usually causes troubles (e.g. FFmpeg with librtmp). It could also help with some license issues when distributing the software is not possible (e.g. FFmpeg non-free).
The GPAC framework includes a player. The player has a lot of dependencies including those generated by Zenbuild. The detection and test of these dependencies is hard and generate a lot of support from the GPAC team.
Build GPAC dependencies on popular platforms (covered by our build infrastructure and TravisCI) from a single script. Allow a GPAC static build with zero dependency.
Basic build skills (make, shell script ; compile, link).
Knowledge on building on different platforms is a plus.