DragonFly is an operating system and environment designed to be the logical continuation of the FreeBSD-4.x OS series. These operating systems belong in the same class as Linux in that they are based on UNIX ideals and APIs. DragonFly is a fork in the path, so to speak, giving the BSD base an opportunity to grow in an entirely new direction from the one taken in the FreeBSD-5 series.DragonFly belongs to the same class of operating systems as other BSD-derived systems and Linux. It is based on the same UNIX ideals and APIs and shares ancestor code with other BSD operating systems. DragonFly provides an opportunity for the BSD base to grow in an entirely different direction from the one taken in the FreeBSD, NetBSD, and OpenBSD series.
DragonFly includes many useful features that differentiate it from other operating systems in the same class. The most prominent one is HAMMER, our modern high performance filesystem with built-in mirroring and historic access functionality. Virtual kernels provide the ability to run a full-blown kernel as a user process for the purpose of managing resources or for accelerated kernel development and debugging.
The kernel makes extensive use of tokens as a synchronization mechanism; tokens are inherently deadlock-free and easily composable. The use of soft token locks results in less cross-subsystem pollution and more maintainable code, both of which allow us to parallelize the system with less effort compared to other kernels, which primarily use hard mutex locks. DragonFly is uniquely positioned to take advantage of the wide availability of affordable Solid Storage Devices (SSDs), by making use of swap space to cache filesystem data and meta-data.
This feature, commonly referred to as "swapcache", can give a significant boost to both server and workstation workloads, with a very minor hardware investment. The DragonFly storage stack comprises robust AHCI drivers, stable device names via DEVFS and a partial implementation of Device Mapper for reliable volume management and encryption. Some other features that are especially useful to system administrators are variant symlinks (i.e. symlinks that are resolved at runtime based on user-specific or system-wide variables) and a performant and scalable TMPFS implementation. Our system makes pervasive use of NULLFS mounts, which allow the administrator to make arbitrary parts of the filesystem hierarchy visible in other locations with virtually no overhead.
A major crux of any open source operating system is third party applications. DragonFly leverages the pkgsrc system to provide thousands of applications in source and binary forms. These features and more band together to make DragonFly a modern, useful, friendly and familiar UNIX-like operating system.
The DragonFly BSD community is made up of users and developers that take pride in an operating system that maintains challenging goals and ideals. This community has no reservation about cutting ties with legacy when it makes sense, preferring a pragmatic, no-nonsense approach to development of the system. The community also takes pride in its openness and innovative spirit, applying patience liberally and always trying to find a means to meet or exceed the performance of our competitors while maintaining our trademark algorithmic simplicity.
The ultimate goal of the DragonFly project at its inception was to provide native clustering support in the kernel. This type of functionality requires a sophisticated cache management framework for filesystem namespaces, file spaces and VM spaces. These and other features eventually culminate in the ability to allow heavily interactive programs to run across multiple machines with cache coherency fully guaranteed in all respects. This also requires being able to divide resources, including the cpu by way of a controlled VM context, for safe assignment to potentially unsecured third-party clusters over the internet. This original design direction, although no longer the primary goal of the DragonFly BSD project, has influenced many of the design decisions made in the intervening years. While full cache coherency is no longer a top level goal, filesystem coherency is, and that direction continues to guide the project in a number of ways.
DragonFly has been going through rapid and ever increasing development since the fork in 2003. This work has included the simplification and general cleanup of the majority of the kernel subsystems. This work was originally intended to support single system image clustering, but has had the effect of making the kernel much more reliable, understandable and easily maintainable. One of the fundamental synchronization concepts that DragonFly uses throughout the kernel, the token, lends itself directly to ease of maintenance and understandability of the kernel.
During the first major phase of the project, which lasted until early 2007, the DragonFly project focused on rewriting most of the major kernel subsystems to implement required abstractions and support mechanics for the second phase of the project, which at the time was intended to be single system image clustering. This involved a great deal of work in nearly every subsystem, particularly the filesystem APIs and kernel core. During this time a paramount goal was to keep the system updated with regards to the third party applications and base system utilities needed to make any system usable in production. This resulted in the adoption the pkgsrc system for management of all non-base-system third-party applications in order to pool our resources with other BSD projects using the system.
In the 2007-2008 time-frame a new filesystem called HAMMER was developed for DragonFly. HAMMER saw its first light of day in the July 2008 2.0 release. This filesystem has been designed to solve numerous issues and to add many new capabilities to DragonFly, such as fine-grained history retention (snapshots), instant crash recovery, and near real-time mirroring. The Hammer filesystem is also intended to serve as a basis for the clustering and other work that makes up the second phase of the project.
Recently, many developers have focused on SMP scalability while others have put an emphasis on new feature development and driver porting. Further and more up to date information on the project goals and status are available on this website, and discussion of the project is possible on a variety of newsgroups, mailing lists and IRC.
DragonFly BSD was originally forked from FreeBSD 4.8 in June of 2003, by Matthew Dillon. The project is "the logical continuation of the FreeBSD 4.x series", as quoted in Matthew Dillon's announcement.
Please keep in mind that major modifications have been made to nearly the entire DragonFly kernel relative to the original FreeBSD-4.x fork. Significant changes have been made to every kernel subsystem, as a consequence this list is constrained to the largest, most user-visible changes unique to DragonFly.
- The scheduler abstraction has been split up into two layers. The LWKT (Light Weight Kernel Thread) scheduler is used by the kernel to schedule all executable entities. The User Thread Scheduler is a separate scheduler which selects one user thread at a time for each cpu and schedules it using the LWKT scheduler. Both scheduler abstractions are per-cpu but the user thread scheduler selects from a common list of runnable processes.
- The User Thread Scheduler further abstracts out user threads. A user process contains one or more LWP (Light Weight Process) entities. Each entity represents a user thread under that process. The old rfork() mechanic still exists but is no longer used. The threading library uses LWP-specific calls.
- The kernel memory allocator has two abstracted pieces. The basic kernel malloc is called kmalloc() and is based on an enhanced per-cpu slab allocator. This allocator is essentially lockless. There is also an object-oriented memory allocator in the kernel called objcache which is designed for high volume object allocations and deallocations and is also essentially lockless.
- DEVFS is the DragonFly device filesystem. It works similarly to device filesystems found on other modern unix-like operating systems. The biggest single feature is DEVFS's integration with block device serial numbers which allows a DragonFly system to reference disk drives by serial number instead of by their base device name. Thus drives can be trivially migrated between physical ports and driver changes (base device name changes) become transparent to the system.
- VKERNEL - DragonFly implements a virtual kernel feature for running DragonFly kernels in userland inside DragonFly kernels. This works similarly to Usermode Linux and allows DragonFly kernels to be debugged as a userland process. The primary use is to make kernel development easier.
- NFS V3 RPC Asynchronization - DragonFly sports a revamped NFSv3 implementation which gets rid of the nfsiod threads and implements a fully asynchronous RPC mechanic using only two kernel threads. The new abstraction fixes numerous stalls in the I/O path related to misordered read-ahead requests.
- Disk I/O scheduler framework (dsched) - DragonFly implements a disk scheduler framework controlled by a set of sysctls and a fair-queue disk scheduler module. The ioprio utility may then be used to control I/O priorities on a process-by-process basis.
HAMMER is the DragonFly filesystem, replacing UFS. HAMMER supports up to an Exabyte of storage, implements a fast UNDO/REDO FIFO for fsync(), recovers instantly on boot after a crash (no fsck), and implements a very sophisticated fine-grained historical access and snapshot mechanic. HAMMER also sports an extremely robust streaming, queueless master->multiple-slave mirroring capability which is also able to mirror snapshots and other historical data.
All non-temporary HAMMER filesystems in DragonFly automatically maintain 60 days worth of 1-day snapshots and 1-day worth of fine-grained (30-second) snapshots.
HAMMER is designed to accommodate today's large drives.
NULLFS - NULL Filesystem Layer
A null or loop-back filesystem is common to a number of operating systems. The DragonFly null filesystem is quite a different animal. It supports arbitrary mount points that do not loop, a problem on other operating systems, making it extremely flexible in its application. It is also extremely fast and reliable, something that few other operating systems can claim of their null filesystem layers.
TMPFS - Temporary FileSystem VFS
Originally a NetBSD port the guts have been radically adjusted and carefully tuned to provide a low-contention read path and to directly tie the backing store to the VM/paging system in a way that treats it almost like normal memory. Only memory pressure will force tmpfs data pages into swap.
TMPFS replaces MFS and MD (for post-boot use).
SWAPCACHE - Managed SSD support
This DragonFly feature allows SSD-configured swap to also be used to cache clean filesystem data and meta-data. The feature is carefully managed to maximize the write endurance of the SSD. Swapcache is typically used to reduce or remove seek overheads related to managing filesystems with a large number of discrete inodes. DragonFly's swap subsystem also supports much larger than normal swap partitions. 32-bit systems support 32G of swap by default while 64-bit systems support up to 512G of swap by default.
VARIANT SYMLINKS
Variant (context-sensitive) symlinks give users, administrators and application authors an extremely useful tool to aid in configuration and management. Special varsym variables can be used within an otherwise conventional symbolic link and resolved at run-time.
PROCESS CHECKPOINTING
Processes under DragonFly may be "checkpointed" or suspended to disk at any time. They may later be resumed on the originating system, or another system by "thawing" them.
DNTPD - DragonFly Network Time Daemon
DragonFly has its own from-scratch time daemon. After pulling our hair out over the many issues with open source time daemons we decided to write one ourselves and add new system calls to support it. DNTPD uses a double staggered linear regression and correlation to make time corrections. It will also properly deal with network failures (including lack of connectivity on boot), duplicate IPs resolved by DNS, and time source failures (typically 1 second off) when multiple time sources are available. The linear regression and correlation allows DNTPD to make rough adjustments and frequency corrections within 5 minutes of boot and to make more fine-grained adjustments at any time following when the linear regression indicates accuracy beyond the noise floor.
DMA - DragonFly Mail Agent
The DragonFly Mail Agent is a bare-bones (though not so bare-bones any more) mail transfer and mail terminus SMTP server which provides all the functionality needed for local mail delivery and simply remote mail transfers. A more sophisticated MTA can then be thrown onto an installed system via pkgsrc.
Updates (via Distrowatch):
Matthew Dillon has announced the release of DragonFly BSD 2.10.1, a BSD operating system originally forked from FreeBSD 4 series: "The DragonFly BSD 2.10.1 release is now available. This release sports significant compatibility and performance improvements and many new features. Big-ticket items: this release supports a much larger variety of hardware and multiprocessor systems than previous releases, thanks to updates of ACPI and APIC and ACPI interrupt routing support; Hammer volumes can now deduplicate volumes overnight in a batch process and during live operation; Packet Filter (pf) was updated to a version based upon OpenBSD 4.4; DragonFly now uses GCC 4.4 as the default system compiler, and is the first BSD to take that step; significant performance gains over previous releases...."
Read the release notes for detailed information about the changes and features in this version.
Download (MD5, mirror list): dfly-i386-2.10.1_REL.iso.bz2 (185MB), dfly-x86_64-2.10.1_REL.iso.bz2 (190MB).
Recent releases:
• 2011-04-27: BSD Release: DragonFly BSD 2.10.1
• 2010-10-30: BSD Release: DragonFly BSD 2.8
• 2010-04-06: BSD Release: DragonFly BSD 2.6.1
• 2009-09-16: BSD Release: DragonFly BSD 2.4
• 2009-04-30: BSD Release: DragonFly BSD 2.2.1
• 2009-02-17: BSD Release: DragonFly BSD 2.2
Screenshots.
Custom Search
If you liked this article, subscribe to the feed by clicking the image below to keep informed about new contents of the blog:
Posts
