Performance engineering on Ubuntu leaps forward with frame pointers by default in Ubuntu 24.04 LTS

This post was co-authored by Oliver Smith, Ubuntu Product Manager, and Jon Seager, VP Engineering at Canonical.

Ubuntu’s long standing reputation for performance is rooted in our commitment to delivering the latest code, kernels and compilers. This commitment ensures that developers have access to the fastest and most secure environment for their applications. Today Canonical is raising the bar for performance and observability in all of our products. 

In collaboration with Polar Signals we have committed that beginning with Ubuntu 24.04 LTS, our GNU Compiler Collection (GCC) package will enable frame pointers by default for 64-bit platforms. All packages in Ubuntu, with very few exceptions, will be rebuilt with frame pointers enabled, making them easier to profile and subsequently optimise.

“I’ve enabled frame pointers at huge scale for Java and glibc and studied the CPU overhead for this change, which is typically less than 1% and usually so close to zero that it is hard to measure. Frame pointers allow more complete CPU profiling and off-CPU profiling. The performance wins that these can provide far outweigh the comparatively tiny loss in performance. Ubuntu enabling frame pointers by default will be a huge win for performance engineering and the default developer experience”. said Brendan Gregg, computer performance expert and Intel Fellow.

Precision profiling with frame pointers

The first and most important tool for performance engineers is the profiler; a new collaboration between Polar Signals and Canonical evolved from a shared vision to improve the performance of applications worldwide. By ensuring that Ubuntu’s compiler enables frame pointers by default, we streamline profiling and debugging processes.

Often overlooked, frame pointers play a crucial role in simplifying the stack tracing process during profiling. Enabling them by default in compiled binaries ensures that developers have immediate access to clearer and more accurate data for performance analysis, with no additional configuration.

There is a small performance penalty associated with the change, but in cases where the impact is high (such as the Python interpreter), we’ll continue to omit frame pointers until this is addressed. Our analysis suggests that the penalty on 64-bit architectures is between 1-2% in most cases. We will not make this change on 32-bit architectures where the penalty is higher.

Enabling such trivial access to performance insights, combined with a commitment to investing in the performance of the platform from Canonical, will facilitate global performance improvements for Ubuntu and the workloads it supports.

Detailed performance benchmarks will be provided in a follow up post closer to the release of Ubuntu 24.04 LTS.

Community and developer impact

More than just a technical adjustment, this is a commitment to the developer community. When performance problems arise in production, developers are often faced with the task of recompiling their applications to include debugging information, then trying to reproduce the situation that caused the software to perform poorly. 

This can be relatively simple for smaller applications, or those with few dependencies, but for large applications with deep dependencies on operating system components, the process becomes incredibly complex.

By enabling frame pointers by default, we’re lowering the barrier to entry for performance profiling and debugging, meaning:

• Simplified Profiling: Dive straight into profiling without worrying about compiler configurations.
• Lower Overhead: Unwinding with frame pointers is significantly cheaper than using DWARF or DWARF-derived information.
• Debugging Accessibility: Even those new to profiling can access high-quality data, democratising the process of performance optimisation. It will allow bcc-tools, bpftrace, perf and other such tooling to work out of the box.
• Ecosystem Benefit: Ubuntu comprises thousands of open source projects, and we’ll work to upstream performance fixes that we find across potentially thousands of packages wherever we can.

Canonical, and the Ubuntu community, can make use of this improved debugging information to diagnose and target performance issues that are orders of magnitude more impactful than the 1-2% upfront cost.

A testament to open collaboration

Our collaboration exemplifies the power of the open source community. By working together, we’re hoping this change will benefit countless developers, organisations and software end-users.

“Canonical is investing heavily in the performance and observability, and it’s clear that Polar Signals are blazing a trail in the practice of continuous profiling, and particularly with their zero-instrumentation eBPF-based Parca Agent. Our decision to enable frame pointers by default in Ubuntu marks a significant step forward in making Ubuntu the most performant and efficient platform for your workloads. I’m really excited to see the impact this has over the coming months and years”. said Jon Seager, VP Engineering at Canonical.

“Hyperscalers have long adopted frame-pointers by default because they recognise that the potential performance impact is negligible compared to the insight they can bring during debugging and troubleshooting. These increased performance insights often enable optimisations that are orders of magnitudes larger than the cost associated with enabling frame pointers. I couldn’t be more excited about Canonical recognising this and bringing it to the masses with Ubuntu 24.04 – this is a huge leap forward for observability and debuggability”. said Frederic Branczyk, CEO/Founder at Polar Signals.

Looking ahead

We’re excited to bring a greatly simplified profiling and debugging experience to the masses in Ubuntu 24.04 LTS, which will see mainstream adoption for years to come. The impact of this change coupled with the follow-on investment in performance engineering at Canonical will result in better quality, and higher performing applications globally.