2020, core development of the Linglong prototype is quietly completed, drawing on sandbox/container technology solutions;
In 2022, as a future core feature of the deepin distribution, Linglong is released with the deepin v23 preview and is initially available;
In 2023, we will donate Linglong project code, Linglong official website, Linglong store and other assets to Open Atom Open Source Foundation, in order to gather more industrial strength, work together to promote the development of Linglong, and accelerate the ecological construction of .......
So what is Linglong? Where does it come from? And where is it going? Next, this article for you to reveal one by one.
Preface: The Evolution of Package Managers
The Linux operating system has always been known for its open source nature and flexibility, and the most critical part of making a Linux system able to install and run the required software without any problems is the package manager.
As the name suggests, a Linux package manager is a tool for installing, updating, and uninstalling software packages on the Linux operating system. Its history dates back to the 1990s, when Linux was in its infancy and software installations had to be done manually by downloading the source code and compiling it. compilation, a tedious and difficult task for non-technical users.
This led to the creation of dpkg and rpm, which were still inconvenient to use because they did not automatically resolve dependencies.
It was not until the release of Debian's apt and Red Hat's up2date that the usability of package managers was greatly improved. They use an algorithm called a "dependency resolver" that automatically resolves dependencies between packages, simplifying the process of installing and upgrading software. This, on the other hand, adds a great deal of complexity, and maintainers need to be very careful to deal with the "dependency hell" that can lead to package system failures.
There are also many other package managers, such as yum, portage, and pacman. This variety of package managers gives users more choices, but the downside is significant: their packages are not interoperable, which means that a piece of software may need to be repackaged if it is to be used on other distributions.
With the Linux kernel's support for containers and the birth of Docker, a number of container-minded package managers such as Snap and Flatpak have begun to emerge. This kind of package format is almost completely decoupled from the system environment, and no longer relies on the library files on the system (AppImage is also the same), application distribution began to gradually become easier. However, the problems of high disk and memory usage and extended startup time have not yet been solved.
Explore: "Linglong" was born
Since deepin abandoned Ubuntu as its upstream in 2015 and chose Debian, Ubuntu's non-commercial upstream community, as the basis for its research and development, we've received feedback from many users about package management issues, the most common of which are:
- There are too few applications available on the system, and the versions of the available applications are too old;
- Some applications do not work properly after a system update;
- After obtaining certain applications from other sources and installing them, the package manager does not work properly and even the system cannot continue to be used.
There is a common cause for all these problems: the dependency bindings are too strong. Due to the underlying system library, the application cannot be updated at will, and when there is an interface change in the underlying library, the application needs to be re-adapted in order to work properly.
After realizing these problems, we started to experiment with a new package manager.
- Adapting Snap on deepin: Since Snap has many compatibility issues except for Ubuntu, we gave up.
- Converting some of our homegrown apps to AppImage: AppImage has good portability, and these apps can easily be used on other distributions. However, it doesn't have centralized repository storage and package management, and doesn't provide the same level of sandboxing as Snap and Flatpak, so its security can't be guaranteed, and it's not suitable to be used as the default package management method for the operating system.
- In 2017, deepin followed up the Flatpak format and completed the construction of 100+ packages, but did not continue to adapt due to the large size of the application, excessive disk occupation, slow bug fixing and other reasons.
After a lot of "tossing and turning", we decided to design our own package management system based on our understanding of various types of package managers.
After more than 3 months of technical research, more than 1 year of prototype validation, technical solution improvement and product polishing, finally an advanced solution - "Linglong" was born.
Top-level component relationship diagram
- ll-box: The application sandbox running environment is designed according to OCI standards, utilizing kernel Cgroup and Namespace features to isolate the application from the host environment and limit the use of system resources.
- ll-service/ll-cli: Provides functions such as application sandbox environment creation and system compatibility issue handling. It accomplishes the management of the installation status/running status of applications.
- ll-dbus-proxy/ll-fuse-proxy: provides privilege management functions, including DBus interface and file interface.
- ll-builder: provides a containerized application building environment for developers to build consistent applications on different environments.
- uab/AppBundle: Uniontech Application Bundle, the application package packaging format, provides a binary package format that can be run directly.
- ll-repo-server: Provides package upload, download, information statistics, query, etc. The underlying storage uses OSTree.
Running View
Achievements: solving compatibility problems and significantly improving performance
Previously, the construction of domestic software ecosystem is still immature, software compatibility and security problems occur frequently, and it will take a lot of extra time and resources when packaging and distributing applications for different operating systems.
The appearance of Linglong undoubtedly provides a new idea to solve this problem. Linglong's isolation technology can completely decouple the application from the system, thus completely solving the compatibility problems caused by upgrading between the system and the application and between the application and reducing the number of packaging times when distributing under different operating systems.
Traditional Architecture Vs Linglong Architecture
At present, Linglong's infrastructure has been more perfect, derived from 5 projects, total 9 components.
Project Components Overview
Compared to other similar package formats, Linglong has many advantages in terms of startup speed and resource consumption:
- Uses a non-full runtime (host system + Runtime) with a smaller overall size;
- Due to the reuse of libraries on the host system, you can use part of the library files that have been loaded into memory, the startup speed will be faster, and the same application under the lingerie startup speed improvement is significant;
- Provide development library hosting service, similar to NuGet, which is convenient for developers to develop;
- Support Rootless (unprivileged) sandbox.
Package Size Statistics
Package startup time statistics
In the latest version of deepin v23, more than 10 Linglong format self-developed applications have been pre-installed. In Linglong Web Store, there are more than 120 commonly used apps, such as QQ, WeChat, NetEase Cloud Music, Xunlei, etc. The accumulated downloads have reached 40w+.
Future: Contribute to the healthy development of OS software package ecology
In the future, we will further enhance and optimize Linglong in terms of permission control, user interaction and the number of available software.
1. Permission Control
At present, Linglong has a single permission to access files, it can only handle the mounting of directories before the application is started, and the unmounted directories can not be accessed by the application after it is started.
In the future, it will support the dynamic control of file access rights, which can be managed no matter whether the application is started or not, meanwhile, the control center will synchronously adapt to Linglong's rights control and provide the application rights management interface.
2、User Interaction
At present, the update of Linglong application requires users to update manually on command line, which requires certain Linux foundation. And when there is a problem with the software package, it is impossible to query the information of the build source directly, such as the hash value of the git project.
In the future, the app store will support linger app updates. At the same time, it supports traceability, which enables developers to quickly query the hash value of the source file used by the software package, making it easier to trace and solve problems.
3、Package Ecology
Eco-construction needs everyone's joint efforts, we have started to develop relevant software package conversion tools, which can easily convert the existing deb, appimage and other formats into Linglong apps. At the same time, we are also promoting the adaptation of Linglong by the existing cooperative software vendors.
Instead of walking alone, it is better to walk together, ecological construction needs everyone's joint efforts. Previously, deepin open source community and Beijing University of Aeronautics and Astronautics (BUAA) have carried out summer open source ecological cooperation, and many students from BUAA have participated in the ecological co-construction.
We sincerely hope that Linglong will be able to solve the problem of difficult distribution of multi-release applications, and also look forward to more interested friends to join us to build the application distribution system, and contribute to the healthy development of the operating system software ecosystem.