Skip to content
forked from BMDan/DFuse

DFuse: Database FUSE - MySQL Tables Exposed as a FS via FUSE

Notifications You must be signed in to change notification settings

taxanotes/DFuse

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

23 Commits
 
 
 
 
 
 
 
 

Repository files navigation

DFuse: Database FUSE.  Your database, presented as a filesystem.

Let's start with FUSE: it's like a shim that sits between the kernel and a userspace program. It means that you can use userspace libraries in the implementation of file systems, which is important in our case.  https://en.wikipedia.org/wiki/Filesystem_in_Userspace#/media/File:FUSE_structure.svg gives you an idea of how it works.

So what's happening with dfuse is that is that we change the semantics of a database into a 1:1 representation on the filesystem via FUSE.  A table in SQL becomes a directory on the filesystem.  Each "file" in that directory is named by the primary key--which makes sense, since a primary key is already guaranteed to be unique, just like a filename, so we can easily map back and forth between the two.

But what makes this work is that this is not a static dump of the data; this is a real-time, dynamic view into the database.  Every "cat", every "ls", every "tar" or "git" is actually a "SELECT" (and usually several SELECTs).  This means you can check the whole database (or any portion thereof) into version control, right alongside every other file that makes up the site.  And since you're using the same revision control tools and systems you use for your code, you can use all your knowledge of how to use them to manage this database data, as well.  Further, because you're using one system to manage both types of data, you can tag a release in a meaningful way; realistically, a release is both code and data, and dfuse allows you to integrate them both in a single SCCM.  Lastly, because the representation fits with the semantics that SCCM tools expect, you're only storing diffs of the DB, not the entire DB, and since you can exclude tables or data that aren't relevant, this means you can check the DB into your SCCM more often than you could if you were simply dumping mysqldumps or even [bg]zip'd mysqldumps in there, without running short on disk space.

As a bonus, because the semantics by means of which we refer to a given piece of data uniquely identify it, the system has the ability to support write operations, as well.  So not only could you check your database in to version control, you could also check it back out (i.e., deploy the entirety of a site, DocRoot and DB both, by a simple "svn co" or similar).  That said, as of this version, the FS->DB write capabilities are quite rudimentary.

As it stands, however, I think it's already pretty darn powerful; it allows you to do things like checking the site in before presenting it to the customer, then, when the customer calls back five minutes later because the entire site suddenly lost all its CSS, running a simple diff in order to determine what they changed.  Hint: it's in the variables table.  ;)

The other nice part about the way the code is written is that it's exceedingly flexible.  If you don't care about Drupal's cache_* tables (and generally you wouldn't), just don't map them onto directories.  If you have something where the primary key is composed of multiple columns, just create a unique, two-way mapping between them (from something as simple as CONCAT(column1,"-",column2) to something terribly complex).  The goal is that dfuse itself should be like a good UNIX command: simple, but spectacularly good at the one thing it does.  Intelligence is then added to it not by modifying it to support a particular use case, but simply by giving it the right parameters or execution context.

About

DFuse: Database FUSE - MySQL Tables Exposed as a FS via FUSE

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • C 98.0%
  • Shell 2.0%