Information for students
To learn more about GPAC, you can find information about the project history, some metrics, and a short description.
As indicated on KDE’s GSoC ideas page:
If you wish to submit a proposal based on these ideas, you may wish to contact the developers and find out more about the particular suggestion you’re looking at this page.
Being accepted as a Google Summer of Code student is quite competitive. Accepted students typically have thoroughly researched the technologies of their proposed project and have been in frequent contact with potential mentors. Simply copying and pasting an idea here will not work. On the other hand, creating a completely new idea without first consulting potential mentors is unlikely to work out.
If you have any question, please contact gsoc@gpac.io.
Ideas
Idea 1 : More support for hardware video
Description
GPAC already has native support for hardware video decoding through VideoToolbox (OSX, iOS), NVDec (Windows/Linux) and MediaCodec (Android).
The aim here is to include support for other video decoding APIs (VAAPI, VDPAU, DxVA) in a first time. The second part aims at integrating hardware video encoding in GPAC with the same APIs.
Expected results
Hardware decoding and encoding in place for various platforms.
Required skills/knowledge
Good multimedia knowledge, strong C programming skills.
We can teach the trainee about the GPAC filter architecture specifics.
Mentor:
Jean Le Feuvre (jean.lefeuvre@telecom-paris.fr)
Backup Mentor: Romain Bouqueau (romain.bouqueau@gpac.io)
Idea 2 : GPAC remote monitoring
Description
GPAC already has support for the great Remotery project, helping visualization of basic information of a filter session (CPU usage, thread usage).
The aim here is to extend the simple remotery viewer and underlying messages to provide more information on various filters in the graph such as processing speeds, packets I/O, buffer states, filter graph information (this could even include live “screenshots” of various streams in the graph). The second part would involve talking back to the filter graph to issue commands, bypass filters, etc …
Expected results
Enhanced visualization in the browser of a GPAC session.
Required skills/knowledge
Simple multimedia knowledge, strong JavaScript/Web programming skills.
We can teach the trainee about the GPAC filter architecture specifics.
Mentor:
Jean Le Feuvre (jean.lefeuvre@telecom-paris.fr)
Backup Mentor: Romain Bouqueau (romain.bouqueau@gpac.io)
Idea 3 : Extend GPAC JS APIs
Description
GPAC comes with support for JavaScript and ES2020 through QuickJS. This comes with a set of Web APIs (XmlHttpRequest, WebGL) and custom APIs around libgpac allowing JS-based media processing filters. We however miss a view simple APIs and some basic glue code to run existing three.js or babylon JS sample goal.
The aim here is first to idendify all missing APIs and glue code these libraries depend on, propose an incremental approach towards filling these gaps and extend the set of Web APIs of GPAC in order to run 3D libraries such as three.js or babylon.js within a GPAC scripting context to simplify filter development.
Expected results
A GPAC filter running a simple to complex three.js and/or babylon.js demo.
Required skills/knowledge
Good multimedia knowledge, strong C/JavaScript/Web programming skills.
We can teach the trainee about the GPAC filter architecture specifics.
Mentor:
Jean Le Feuvre (jean.lefeuvre@telecom-paris.fr)
Backup Mentor: Romain Bouqueau (romain.bouqueau@gpac.io)
Idea 4 : More support for 360 videos
Description
Google has added a documented XML metadata to describe the orientation and projection of their 360 videos, Facebook is doing similar things and MPEG has just published the OMAF standard.
GPAC already has native support for 360 videos from equirectilinear projections (video) and some cube map support for images QTVR, both including user navigation.
The aim is to include support for Google/Facebook/OMAF signaling within GPAC, both at the packaging side (MP4Box) and at the client side. Some extensions would be to add support for Youtube URLs in GPAC or support more projections and formats from other popular 360 video providers (Facebook, OMAF…).
Expected results
Play 360 videos from Youtube or other platforms with different projections.
Required skills/knowledge
Basic multimedia knowledge.
We can teach the trainee about the ISOBMF/MP4 muxer specifics.
Mentor:
Jean Le Feuvre (jean.lefeuvre@telecom-paris.fr)
Backup Mentor: Romain Bouqueau (romain.bouqueau@gpac.io)
Description
With the recent growth of 360 video applications, there is a need to provide immersive spatialized audio experience in media players. Two main approaches are used in the industry: object-based coding using multichannel setups, and ambisonic approaches.
GPAC already has support for some spatialization tools and multichannel audio formats.
The aim is to include support for libstatialaudio in GPAC, and propose packaging guidelines for ambisonic content.
Expected results
Spatialized playback of ambisonic sound within GPAC.
Required skills/knowledge
C knowledge, basic multimedia knowledge.
We can teach the trainee about the ISOBMF/MP4 muxer specifics.
Mentor:
Jean Le Feuvre (jean.lefeuvre@telecom-paris.fr)
Backup Mentor: Romain Bouqueau (romain.bouqueau@gpac.io)
Idea 6 : Simplify Testing for developers
Description
GPAC provides many tools, mostly playback, packaging and networking ones, with a large number of tests covering many aspects of these tools. At the current time, there is no way for a developper to know which tests should be run after a code modification, and the entire test suite should be re-run in order to check for any regression. This process is quite long and is therefore usually not done by the developer, but by the build bots.
The aim is to provide tools for the developer to simplify testing and code checking before any commits, effectively reducing the number of tests to be run to validate a commit.
Expected results
Faster, context-dependent test suite execution for GPAC developers.
Required skills/knowledge
Basic C knowledge, BASH and git knowledge.
Mentor:
Jean Le Feuvre (jean.lefeuvre@telecom-paris.fr)
Backup Mentor: Romain Bouqueau (romain.bouqueau@gpac.io)