Monthly Newsletter
WHAT'S RTC.ON NEWSLETTER ABOUT?
Each month, our team will get together and give you a selection of multimedia dev content.
Grab yourself a coffee (or any drink of preference), sit back & enjoy the next edition of RTC.ON Newsletter – we hope you'll like it!
RTC.ON 2024 TALKS ARE NOW OUT
Kasia Smoleń
MARKETING SPECIALIST @ SOFTWARE MANSION
Catch up on all RTC.ON 2024
RTC.ON 2024 may have wrapped up a while ago, but it’s still fresh in our minds. To ensure you can easily rewatch your favorites or discover interesting talks, we’ve put together a YouTube playlist featuring every session, so you can explore topics like WebRTC, QUIC, real-time AI, VLMs, Elixir WebRTC, low-latency voice assistants, and more at your convenience.
The RTC.ON after movie is out
They say hosting a conference in an office isn’t easy, but with the right vibe and the right crowd, anything’s possible! Three days of WebRTC, Computer Vision & AI packed into one great after-movie. Catch the highlights, the vibe, and everything you missed (or want to see again) from RTC.ON 2024. And if you’re wondering what next year might look like, we’ve got big plans – RTC.ON 2025 is on the way, and it’s only going to get better.
G-STREAMER CONF & DEMUXED RECAP
Mateusz Front
SOFTWARE ENGINEER @ SOFTWARE MANSION
Catch up on all RTC.ON 2024
Last month I visited two media streaming conferences: GStreamer Conf and Demuxed. It was my first time participating in either of them, so I was particularly curious about what I would experience. Long story short: it was well worth it!
GStreamer Conf took place in Montreal, Canada. Even though I don't use GStreamer daily, Membrane is heavily inspired by GStreamer, so it was a great opportunity to learn more about it. I was surprised to see how many GStreamer use cases cover the user-facing side of things, like embedded, mobile and desktop apps. As Membrane focuses on the server side, I learned a lot about the complexity I only deal with sometimes. But there were many familiar topics too, like incorporating QUIC, Vulkan video coding, and improving WebRTC implementation.
The GStreamer community works hard on rewriting their plugins to Rust, which allows revisiting older, widely used tech, like RTP, and making it more standard-compliant. I also had an opportunity to deliver a talk about Membrane, explaining our approach and the different set of tradeoffs we chose. All the talks are available on the X.Org Foundation YouTube.
From Montreal, I headed to San Francisco to meet fellow developers from Software Mansion and take part in Demuxed. It's probably the biggest dev-centric media streaming conference in existence, with most of the industry's big players involved. The talks covered Twitch incorporating enhanced RTMP, standardization of C2PA and CMCDv2, AV1 adoption, updates on Media Over Quic and many more topics. Apart from enjoying the talks, we had an opportunity to meet the best experts out there: standard creators, browser developers, codec implementors and other media devs - all in one place. Some of them promised to show up at RTC.ON 2025 - let's see!
MULTIMEDIA MUST-READS
Piotr Wodecki
SOFTWARE ENGINEER @ SOFTWARE MANSION
Fork
Luke Curley, a staff software engineer at Discord and a big Media over QUIC fan and contributor, recently dropped a bombshell announcement that he is forking the MoQ specification. In this blog post, he shares his frustrations regarding his work at IETF and describes the plan going forward. I highly recommend this very short read.
Writing an MP4 Muxer for Fun and Profit
Dennis, an OBS core contributor, discusses the development of the "Hybrid MP4" output format in OBS 30.2, designed to combine the reliability of MKV with the compatibility of MP4. He discusses the structure of the MP4 file format, highlighting the issues with standard MP4 and fragmented MP4, particularly in terms of data loss during live recordings. The Hybrid MP4 format mitigates these problems by initially recording as a fragmented file and then finalizing it to resemble a regular MP4, allowing for easy playback and recovery.
Pushing the Frontiers of Audio Generation
This article discusses advancements in audio generation technologies at Google that enable more natural and engaging interactions with digital assistants and AI tools. The authors go over the development of models capable of producing high-quality, multi-speaker dialogue, which are now integrated into various Google products.
Critical Chrome Security Update: Patch for Out-of-Bounds & WebRTC Vulnerability
Google has released a critical security update for Chrome, addressing serious vulnerabilities on Windows, Mac, and Linux. The update targets two significant flaws: CVE–2024–10487, an out-of-bounds write vulnerability in the Dawn graphics system, and CVE–2024–10488, a use-after-free issue in WebRTC. Both vulnerabilities could allow attackers to execute arbitrary code or crash the browser, with the out-of-bounds flaw posing a particularly high risk.
DTLS "ClientHello" Race Conditions in WebRTC Implementations
On the topic of security, researchers at Enable Security uncovered a common security flaw in server-side WebRTC implementations, which can lead to Denial of Service attacks. The study shows a gap between media consent verification and the DTLS handshake, allowing malicious packets to disrupt legitimate communications. It emphasizes the need for stricter source verification to enhance security, particularly in media servers, while noting that browsers are generally secure.
WHAT'S NEW IN SOFTWARE MANSION MULTIMEDIA TEAMS
Kasia Smoleń
MARKETING SPECIALIST @ SOFTWARE MANSION
Fishjam Cloud early access is open – come & join us
We recently introduced Fishjam Cloud – a platform designed to simplify building real-time communication-based products. Fishjam Cloud lets you launch multimedia infrastructure with a couple of clicks and create an up-and-running app with a video chat in just few lines of code. Got a React Native app that could use a touch of multimedia magic? Our early access tests are still underway, so check the link below and join them.
React Rustification for live streaming
In recent years, Rust has become a vital part of the JavaScript ecosystem via tools like SWC, or Turbopack. Live Compositor is our attempt to do the same for live streaming – it harnesses Rust and WebGPU for top-notch performance and features a JavaScript SDK for stream control. Here’s how it works: you write your React components, and the SDK executes that code in a Node.js runtime. When a component re-renders, the SDK sends an update request to the Live Compositor server, which handles the actual video rendering and stream management. Check Wojtek Kozyra's article article to dive deeper.
New WHIP/WHEP SDK for React Native is now available
Implementing WHIP and WHEP protocols for React Native can help build streaming apps that rely on WebRTC. To make the task easier, our team has created an SDK supporting WHIP/WHEP protocols for React Native, Android, and iOS. With the react-native-whip-whep package, you can easily set up a WebRTC PeerConnection, configure stream settings such as resolution and audio/video preferences, and manage media streaming in just a few steps. Learn more in Justyna Gręda’s article.
ELIXIR STREAM WEEK IS OVER –
HERE'S A QUICK RECAP
Karolina Kulig
MARKETING MANAGER @ SOFTWARE MANSION
Elixir Stream Week
WOAH, that was an exciting week! Five days filled with Elixir broadcasts from top Elixir minds, including:
- José Valim, creator of Elixir
- Mateusz Front, creator of the Membrane Framework
- Chris McCord, creator of Phoenix
- Jonatan Kłosko, creator of Livebook
- Filipe Cabaço, Elixir dev at Supabase
All of this live-streamed using Elixir WebRTC! It was the first time we organized an initiative of this kind & we couldn't be happier about how it turned out.
Here's a quick sum-up of ESW in numbers:
- Almost 700 people signed up for ESW updates,
- We had a peak of 409 viewers during José's stream,
- Elixir Stream Week was watched from 42 countries,
- The whole event was running on a single 32 cores, 64 threads machine,
- We had ~30% of CPU usage and almost 700 Mbps of total throughput at the heaviest moment.
For those interested in more content about Elixir Stream Week – i.e. how it was organized, what went well, what could have gone better, some technical details behind the streams – we'll be doing a longer blogpost with full ESW breakdown. And, of course, we'll let you know here once it's up :)
Missed ESW streams? No worries – you can now watch back all of them on our YouTube:
Thanks for making it this far!
Happy streaming :)