FAT integration
This commit is contained in:
189
content/blog/MOP2/fatfs-integration.adoc
Normal file
189
content/blog/MOP2/fatfs-integration.adoc
Normal file
@ -0,0 +1,189 @@
|
||||
= FAT filesystem integration into MOP2
|
||||
Kamil Kowalczyk
|
||||
2025-11-19
|
||||
:jbake-type: post
|
||||
:jbake-tags: MOP2 osdev
|
||||
:jbake-status: published
|
||||
|
||||
Hello, World!
|
||||
|
||||
I would like to present to you FAT filesystem integration into the MOP2 operating system! This was
|
||||
possible thanks to `fat_io_lib` library by UltraEmbedded, because there's no way I'm writing an
|
||||
entire FAT driver from scratch ;).
|
||||
|
||||
Source for `fat_io_lib`: https://github.com/ultraembedded/fat_io_lib
|
||||
|
||||
== Needed changes and a "contextual" system
|
||||
|
||||
Integrating fat_io_lib wasn't so straightforward as I thought it would be. To understand the problem
|
||||
we need to first understand the current design of MOP2's VFS.
|
||||
|
||||
=== MOP2's VFS explained
|
||||
|
||||
The VFS (ie. Virtual File System) is a Windows/DOS-style labeled VFS. Mountpoints are identified by
|
||||
a label, like for eg. `sys:/boot/mop2` or `base:/scripts/mount.tb`, which kinda looks like
|
||||
`C:\Path\To\File` in Windows. I've chosen this style of VFS over the UNIX kind, because it makes more
|
||||
sense to me personally. A mountpoint label can point to a physical device, a virtual device (ramsd)
|
||||
or a subdevice/partition device. In the UNIX world on the other hand, we have a hierarchical VFS, so
|
||||
paths look like `/`, `/proc`, `/dev`, etc. This is a little confusing to reason about, because you'd
|
||||
think that if `/` is mounted let's say on a drive `sda0`, then `/home` would be a home directory
|
||||
within the root of that device. This is not always the case. `/` can be placed on `sda0`, but `/home`
|
||||
can be placed on `sdb0`, so now something that looks like a subdirectory, is now a pointer to an
|
||||
entirely different media. Kinda confusing, eh?
|
||||
|
||||
=== The problem
|
||||
|
||||
So now that we know how MOP2's VFS works, let's get into low-level implementation details. To handle
|
||||
mountpoints we use a basic hashtable. We just map the label to the proper underlying file system
|
||||
driver like so:
|
||||
|
||||
[source]
|
||||
----
|
||||
base -> Little FS
|
||||
uhome -> Little FS
|
||||
sys -> FAT16
|
||||
----
|
||||
|
||||
Through this table, we can see that we have two mountpoints (base and uhome), which both use Little
|
||||
FS, but are placed on different media entirely. Base is located on a ramsd, which is entirely virtual
|
||||
and uhome is a partition on a physical drive.
|
||||
|
||||
To manage such setup, we need to have separate filesystem library contexts for each mountpoint.
|
||||
A context here means an object, which stores info like block size, media read/write/sync hooks,
|
||||
media capacity and so on. Luckly, with Little FS we could do this out of the box, because it blesses
|
||||
us with `lfs_t` - an instance object. We can then create this object for each Little FS mountpoint.
|
||||
|
||||
.Internals of the VFS mountpoint structure
|
||||
[source,c]
|
||||
----
|
||||
typedef struct VfsMountPoint {
|
||||
int _hshtbstate;
|
||||
char label[VFS_MOUNTPOINT_LABEL_MAX];
|
||||
|
||||
int32_t fstype;
|
||||
StoreDev *backingsd;
|
||||
|
||||
VfsObj *(*open)(struct VfsMountPoint *vmp, const char *path, uint32_t flags);
|
||||
int32_t (*cleanup)(struct VfsMountPoint *vmp);
|
||||
int32_t (*stat)(struct VfsMountPoint *vmp, const char *path, FsStat *statbuf);
|
||||
int32_t (*fetchdirent)(struct VfsMountPoint *vmp, const char *path, FsDirent *direntbuf, size_t idx);
|
||||
int32_t (*mkdir)(struct VfsMountPoint *vmp, const char *path);
|
||||
int32_t (*delete)(struct VfsMountPoint *vmp, const char *path);
|
||||
|
||||
// HERE: instance objects for the underlying filesystem driver library.
|
||||
union {
|
||||
LittleFs littlefs;
|
||||
FatFs fatfs;
|
||||
} fs;
|
||||
SpinLock spinlock;
|
||||
} VfsMountPoint;
|
||||
|
||||
typedef struct {
|
||||
SpinLock spinlock;
|
||||
VfsMountPoint mountpoints[VFS_MOUNTPOINTS_MAX];
|
||||
} VfsTable;
|
||||
|
||||
// ...
|
||||
----
|
||||
|
||||
.Little FS init code
|
||||
[source,c]
|
||||
----
|
||||
int32_t vfs_init_littlefs(VfsMountPoint *mp, bool format) {
|
||||
// Configure Little FS
|
||||
struct lfs_config *cfg = dlmalloc(sizeof(*cfg));
|
||||
memset(cfg, 0, sizeof(*cfg));
|
||||
cfg->context = mp;
|
||||
cfg->read = &portlfs_read; // Our read/write hooks
|
||||
cfg->prog = &portlfs_prog;
|
||||
cfg->erase = &portlfs_erase;
|
||||
cfg->sync = &portlfs_sync;
|
||||
cfg->block_size = LITTLEFS_BLOCK_SIZE;
|
||||
cfg->block_count = mp->backingsd->capacity(mp->backingsd) / LITTLEFS_BLOCK_SIZE;
|
||||
// left out...
|
||||
int err = lfs_mount(&mp->fs.littlefs.instance, cfg);
|
||||
if (err < 0) {
|
||||
ERR("vfs", "Little FS mount failed %d\n", err);
|
||||
return E_MOUNTERR;
|
||||
}
|
||||
|
||||
// VFS hooks
|
||||
mp->cleanup = &littlefs_cleanup;
|
||||
mp->open = &littlefs_open;
|
||||
mp->stat = &littlefs_stat;
|
||||
mp->fetchdirent = &littlefs_fetchdirent;
|
||||
mp->mkdir = &littlefs_mkdir;
|
||||
mp->delete = &littlefs_delete;
|
||||
return E_OK;
|
||||
}
|
||||
----
|
||||
|
||||
So where is the problem? It's in the fat_io_lib library. You see, the creators of Little FS thought
|
||||
this out pretty well and designed their library in such manner that we can do cool stuff like this.
|
||||
The creators of fat_io_lib on the other hand... yeah. Here are the bits of internals of fat_io_lib:
|
||||
|
||||
[source,c]
|
||||
----
|
||||
//-----------------------------------------------------------------------------
|
||||
// Locals
|
||||
//-----------------------------------------------------------------------------
|
||||
static FL_FILE _files[FATFS_MAX_OPEN_FILES];
|
||||
static int _filelib_init = 0;
|
||||
static int _filelib_valid = 0;
|
||||
static struct fatfs _fs;
|
||||
static struct fat_list _open_file_list;
|
||||
static struct fat_list _free_file_list;
|
||||
----
|
||||
|
||||
There's no concept of a "context" - everything is thrown into a bunch of global variables.
|
||||
To clarify: THIS IS NOT BAD! This is good if you need a FAT library for let's say a microcontroller,
|
||||
or some other embedded device. Less code/stuff == less memory usage and so on.
|
||||
|
||||
When I was searching online for a FAT library, I wanted something like Little FS, but to my suprise
|
||||
there are no libraries for FAT designed like this? Unless it's a homebrew OsDev project, of course.
|
||||
I've even went on reddit to ask and got nothing, but cricket noises ;(.
|
||||
|
||||
I've decided to modify fat_io_lib to suit my needs. I mean, most of the code is already written for
|
||||
me anyway, so I'm good.
|
||||
|
||||
The refactoring was quite easy to my suprise! The state of the library is global, but it's all nicely
|
||||
placed in one spot, so we can then move it out into a struct:
|
||||
|
||||
[source,c]
|
||||
----
|
||||
struct fat_ctx {
|
||||
FL_FILE _files[FATFS_MAX_OPEN_FILES];
|
||||
int _filelib_init;
|
||||
int _filelib_valid;
|
||||
struct fatfs _fs;
|
||||
struct fat_list _open_file_list;
|
||||
struct fat_list _free_file_list;
|
||||
void *extra; // we need this to store a ref to the mountpoint to access storage device hooks
|
||||
};
|
||||
----
|
||||
|
||||
[source,c]
|
||||
----
|
||||
// This is what our VfsMountPoint struct from earlier was referencing, BTW.
|
||||
typedef struct {
|
||||
struct fat_ctx instance;
|
||||
} FatFs;
|
||||
----
|
||||
|
||||
I've then gone on a compiler error hunt for about 2 hours! All I had to do is change references for
|
||||
eg. from `_fs.some_field` into `ctx->_fs.some_field`. It was all pretty much brainless work - just
|
||||
compile the code, read the error line number, edit the variable reference, repeat.
|
||||
|
||||
== Why do I even need FAT in MOP2?
|
||||
|
||||
I need it, because Limine (the bootloader) uses FAT16/32 (depending on what the user picks) to store
|
||||
the kernel image, resource files and the bootloader binary itself. It'd be nice to be able to view
|
||||
all of these files and manage them, to maybe in the future for eg. update the kernel image from the
|
||||
system itself (self-hosting, hello?).
|
||||
|
||||
== Fruits of labour
|
||||
|
||||
Here are some screenshots ;).
|
||||
|
||||
image::/img/fatfs-showcase2.png["showcase 2"]
|
||||
image::/img/fatfs-showcase1.png["showcase 1"]
|
||||
Reference in New Issue
Block a user