From 1f13f2025a586195403663d418fb35d7d9a5ed31 Mon Sep 17 00:00:00 2001 From: kamkow1 Date: Wed, 19 Nov 2025 21:56:28 +0100 Subject: [PATCH] FAT integration --- assets/img/fatfs-showcase1.png | Bin 0 -> 4718 bytes assets/img/fatfs-showcase2.png | Bin 0 -> 3036 bytes content/blog/MOP2/fatfs-integration.adoc | 189 +++++++++++++++++++++++ 3 files changed, 189 insertions(+) create mode 100644 assets/img/fatfs-showcase1.png create mode 100644 assets/img/fatfs-showcase2.png create mode 100644 content/blog/MOP2/fatfs-integration.adoc diff --git a/assets/img/fatfs-showcase1.png b/assets/img/fatfs-showcase1.png new file mode 100644 index 0000000000000000000000000000000000000000..53e3f4ddea472e45290385efd666f1625fbbaad5 GIT binary patch literal 4718 zcmZXYd0bNK*2gy_YB@Gprebv}8yrq)mIF>{W}U>;G)1)GB&azi4j76JR%Tk6TA8R3 znG=?hv%xu*Xr_SS1UVF$3gV3BfWF)AJ-zq6?_a>)&u8!Tto8eS*Lt|^X#a}H`SLInMQt5%sV(r%c-cQ$bAE?@fpSIYuZCE;AvOG6- zE;mj(wyab?-suykN7x3Kx5PaqtO7zXSJeRMYEM9Z2NtrGjkq==!DkT^fZrAC$Fntj zvXAm+XJ^45W0rF+afFL+7c3qHVuEAP=k3f&ItvFrMmAdqYZF~YK9QwMOQeQ+|&q&(&L)PYHrbW-eZ+p=$ljW+&y{*&>Tm>8-0!f{oP4u_|CIKhfj7)pUuxh zonF)0)>My7dA06(%#C?(aZ<6k-Bopu!mGbKas{w-jQB??Li!AGk7pmSsqT4Q)}@T< zfoegdWiN7|IyK;8+|)6CiHes-hwPa_MvfLG)NE{F*IPQBnLUPzWh0?xnO614ysEI< z6!D)I#soNKiN+YFt(D2jl}gjS<*jNAuNo8nTv$LN0yaetP8{Z%@2GFoJ2kQYnDKXl z0Ox+lTc{sgOMM9I`kIt;*JuXDWEbzS-STm`**!pevPaaSU%k9>%>4uO5*Nn~x^k3% z{=l;uiqz%6ut3_W{Xu`r6WOyjsE3L+$Gv0G8~qyMuq=J>|$PBA1kfce=4XxvHds;iwcUpbWVc1`8PWh?>=lOU@IAUz6dA$ z3~5x9F2P1>pL?4={~QY>b0vnJ9o>Wnn}EU|L+n5z63AJPMkYXsh3jd6>M=IdD(55% z%gSoqLXgZf*}jliFrR%@)hj7RD)$u*y>D;HSVqi{`x86l#C3OG6r&QF>jL*2fBki= zNw{70kjvg?i2Lis{J0$Uuy!(kS!! zQ`<_~oVpUXmCn)Yj9@zLF%}7<%oER-d3@3PFcs~%XiwKS`2!x8hEd3+=7J z)N6uEvwmnN-mDy-XM$H~IF8_6AO$r@uAz>%T(%R}*WTexSHR%guhb_x-2ODLl79)& znNs4(y%aIu<~^M0nEoX`V$O#i^Z3txuZ2Gnw$MqPs4UAXd$TNcHBG5L7h}T~%nDRO zJ9YAg^37Wkt8g9^I!h&6@nH6K`HagfJZ*hUJLXNe9Fa2Wu`Aey;0Cl#-pbL6IPNx~ z$Z#&01+2DOD**8_dVq|u5MHdl$aq>^>&2Ugi$?>5dTnyw+rk`Xu763?_exQ#L_GeW zWZODH-sY)biw6uZL}se6KC$}3O>7TWbAXSMTlO?OM3+XGaNd! z83;N8>gZQ^>p)@c4#B((_l`By^63W7u9aY`XfNU7=bl8R-9#K1i7>fK6g4Tz>pL2M z^gxOAvc^hUouXFdD)b#`CZzwAO+qr$$7Zm{jh-E*C*(ji#OrB^ zkP!E!LRdlsyJraZl=uro^!cUkG+D!r8_a>+2F)r_eu z(ds|>b}t9K8l$%HM@*~qe~ETR-&+C5C}*i9J;p9dB#|#(%sqieFHg*7&uXVVmg!H2 zCay^`F*P|O9Co{_;nOv=mblM!Ts_d3SlAX8lWG9;uWJR<4Bt;u{B1Bv+Bvrg2w9Dh z5to$Emo*Y_f+iqjV>(#P9O|_WLr;K873D~h)QE+AOf=G0-11{wf}xC9CMShyxB-m7 zl^n-c!@0yIv-0eKKV_@fGpcAOTC#DRw3 zel2n8Ji-J*aQ~SNkoFy#zJTdO4xVP_{Z)eZ;3!z#A>e6!eSM$$)eB)s{4FXD2wt~0 z^WPk|()!8q?zRDfP2}9_v9{GRcZYr7;VLv}inxE%12<30a$dMYMVZExZtGl3FBN0TC!U}i4YmU@Cck6t ztMoRPH4Gi6p9%fT;VdTA@5iu_yZ))$72D=^p1qXEU1}5gvX+0u^7#gJszh!edxk3R3) z+j!+zRqPc3fg0uGthV)7r*+MQdf=U=`sQ0WGm0YWVW9R{*^*NPhiwFy3Gwxb6bTmP zyDQivPSX4dQ=!MyFffYV@j2=O}tjlnHg*ur~QEys+!~t7^*s86jtkqd^z_LoJ)qb;5q~VcNaxg)0eD`dTJs(?wr>2hrPGPJA(RaC* zVOx3RYY|<0Jl-1uJL^7z20^WG=VwE^m(vpp6L$4mXUhR2{>wQ%hz%57pc7XQ8iSoe z7IE|o%rbi>etc#r0BWnd)%_oor1q!|n(c3o#IAb{c)9l%UEs(AD=z(MHtGgijR6Zr z#J*0=Ju33mVrfi&-b4(D4o7$bw<0LH7l2C5f7tvhh^}YQP+@aC*lsgMILQ}fdu0y> zK!)0PUW;Jgfc0-z)-rWG6!oFwZ;gv0(Oo7OpL^)%R;MXpnFLSEcNlf#Y1$e9b;8I4 zTyvt!b~WOOp+nexdm)Mdx=Q%xZlLmdrcChw0+`OflPj1NbQ$TE8ItxgAvs8NCt0Ek z0;Q&jz5idg*AYBG1+CEH*J!ue+RUSAYT`#*1dyX`FIS9}=|7cB!aeP%&5R51bCK&a z2XxyxlO+7c+P26t`|ocErIH}XR!@V}-0wY~jqH6&+&v$Zhb={g;g`C&-wp$|qMg`^ z6pa-7X*bA_z5NlUoRdDBcnz@|o3oFY)MGJpQ$^FN5o2YZT{n7O!otTaYG#j;m^k71Idio{0@c=XKHo z$_De~$_zBOz(>HB)XM;z$xWz#nonp3AvfVn4ofK;{ATG>bJN~8Wq3_n$5x`r1T z_=jMEB7WC%ZW;%vqqHCOo%Vh3_Bomc>5h9}g;6AJz)_?6M99*QXR#9&?p0@)!BmLZBw zg`5|wNU3<*@I=a5XaBtL@2^6JDvfrS(Sr!`k;CkunZ*>`FrP2C)dt+7M?q<=9MLhH zOTM@#l?2S|U`tMxLc^)dCMO_7<5#Uuu?y@q@G;UpONO3g9ZkA?Yiv>3ve$cXRwcw; ztC%c~Q^yWQ{+O6CrNY&Fs+Ic-kGF-{LXUD1EXD!lur` z#g~Pg*xI0h#>8gvQWfN0dkh;+zKN$zD)y(#h!29TG2|5Dr%IDs%5|`&Q8kG1XPD*k zkJY@ejlVU3z~7B+CkIB`56_*jd`y{DVMKtDz=#XOglHAO>&eP6@y`ZU)azv593yPT zY!VOD3@u*h?c_7J%kT|ocxg~@wgv++-A1kx1xbeAF6RcjAUuf{I9l>dI-=mWzK0;C zw1n89%IDd5T57ruY=Ot$(=*6S9TXSZE3m7)6OvEN=i4t#peXp)^~i2wLq~df30%*L zg&*TYoKA`G`{=5V2tElu6eUDpQ(kwnhteaysWbFe^8^I~$hZ|kR=iWl@eTGxqxtAo z_Y}1pq}IuVy%cWSv_I&^8h-53!nZlC@`|!i3m?md-zW?Dnc(-!vN?8O4MA;#MTat^ z^8Y?3{_<(otK^wi^F=^NH93ZMvmFB?!k<WVm;$(=*S-*?U7(vH1e{p|Ql_u%)6c_WKiZcFkEy)127{w>ot6l0Oeq2b%cBuhH0;RF!k zjvi6OS!MA?0#2Z8^!2Ww=qv|8cZDJYjWZG*HVWHI_%xZ_uRJbeFp7I3|>~n;P z_E=jzpAY;Y&r5Q?n`6rIf;iHh_J5YDE)dG}m4?OOwNK4F>ufT2*+iwg}#b`_V3AW228s zg($&a0G7zz+L2;Dm8{iwCh_RUPgZr5J1yv z!Ne1L(Ov0N8Q$2Q07xaxUQWE<{4y-6bSh)^-JbU77I^j_&plm7q=G_jJ}j!}Em+Q( zVtbSc2|jlw&sZY{e6175jtym?d!oUzI_ms)S!uZX#E%IK`tmoVRFXC~x^NbE;T%v& zB}62AuoZxlz2U)@&gsU56#8=Q@=M))K*fevFu@~o>NN@}=yd*hkPm;yv#@0luY(aG zgFXOb=hOY_Ld1D<^`ouo3nf!O4-N7>AD_78LyDq?R7n3`-7yH!3N2TRp#fg#PHU06 z9^hWVnk0SXWlnq%v5t4gX{rwx7Xd`R(@JBpbakS%9ZaG`Z%ZX|!?%I1rM@~X`wQGxmqy}2xSy7m*4UKyBmn5#g}Wc{OB>~uA847WS?^koZO%XUqvTWY z+bu>(+`3kuSMPqV?V7r3rlYpsZ`^TyaQ4OlEmIw%F|vZEask<@rb7E9ZzKpIkI|2clU zWFz*>1kxun*sSUP%z=ePJXB>yH%g6~6+I(Z^u5(}5Pj=;y)EXrB;Wpnys&9i=Tnr%N;qb)ap0}SY(dw=4byl%FmtN_bz&#{`-Xe z(qsgN(>@{AZh#aVwk#aGA3Gu`apH{9^$q7mV%68_WW!P}|LfK5o;Dyh`KR7qL8Ryr}kShzQtn+2t zu~)=-TlBAHe=pi|*mN6@nbp59_+K)@#|oag)F74mI&+z@Qh^Y%LB(AdHTk zPWU*c$75{&tAWMY))`?j0eip0iN_x_y3OO^U?7vXr-mqm!HSF*%n{ zwS-m4g#&KhZ~7Z66VSYFIL%@AbW4Ry2-JPrgfO^Y$l^$w_H*QToa~plM}>`pZ3HEq z>Kokp6^|Fja}HQ7l}wQ99u-sZm8UKz7K5_nWpeAwgEJ=8nx~5!6sBuip=Z@%-kzn< zLljO90MK2UOfLFmt^S%9S@ou*wZO23d>=poPvX^pl`Yfw>kPjC;X|gU9P#Z$4AwTX zZ?F~uq#c{)CG$55LV9=7eute?jfP?@yKaFuMC$(VN#cUq`w&9|-$DiphZk8qO1wJd zr*1_u-}ic#<--7QH65B$i+uVxXsW(mT2xix*6}oJ{FNAE)tN9JbqBM+ukuTggs@v4 z0*%H+?9JpWvXs(A{*XbG^H4&tM{rQA2dqN7=Y6hucM4|_c{oC9e{W(Wt~R9exKyYC z*k0;U{t~Oiu{_rwOpAs;t?(^!dRM5N`=0I92QpwaOV$3;Wa1o3wpwDZzZ70`eM}$@ zHx!)#F+c!Aa6l=Qc3$I)Oh`4*iUC~l4^(k#OP$_R2oN`w%|5S=*-PZvSHGZkn3A2^ zf!>>C4{7D&IBrXdSn1*em4_QGBKq%>mZnzvppki_^1$>CmIrs!5`b zaa3!xj{WO>l;-|)m2m4qvnt%cUTyLa^D6=(TYxe>ynwb`5qY95mo@e`)?p540*#tt zB#;R!oy9wncdQ41pGxNNi54Vr$+EtRBrBVcfcKCNF&11T1-8-ne5cPhySNf*_-{yd zyZk!Sf%}MDvCZa8o#ZNyCltt_ub!x$u&5{}%=@nPjaGdv&(qv`Qx^iKc{S#Iuj&Tf zg?Q`)BiiS|f@0+(VnELr|C2>f30a>+xg8p&`)Xqi$n!bAV_0nN`-%2j#D1l*%^+-U z%^MDgvy(sYGzFpBYYUn7IXakbQTe&&p&m!CZ-%Y!d@xT>J?c^ncT_mU&($se{uUqGX1IdQ9(eM3DVws)~L#cJxyte#BKm9{zPlUnt1#BwI;W*doJQ*`d9(Y)_5un z-@rbfC4xp?Lh2{XuZ^rbBxu1vp3H7NJLuZgW;x8w5t>Mab0_*kk2N|7=^KoZ>W&;2 z&ojB$FEl=+70(GG+n|Gl7rHlie%G~44I;s5zI~v6oyd%{sqV-1tk?t=^T%RJD!7$l`&&tMK+|)G*8^RJ2$EXy$lFJkS&4676r3Q4kP$eYAoIr&@e7{W^FM%!~9^ zWUtywoTq3Ll>YgrTf3$;JEoCSzBxygQorBOx_a=t!XOOsTMG8#Q9GWf)gR*n>J9vc zFXt@!DC+w*j~b?h47o{FzF0%tGXxG+8;qw`Zx42X5GRafw49q<1(17LMmSsp6k$X_or zm?WPG$(+X2D$!Bc^3EY?Pc7`{=i0zztk#^{jO~pKg;9Rdx;ix=<^q?U=)f72!dC0W7lppFLb!an*!JCz+}zWYo+)qtf)HqMKP z(%Bq@22Pup{Lx!i!K^|PzhjF`Qn_*6D5J$-JVep7lWyF{4jUkD-Wo}1C@r|zka0xO z7Upa?J*((pu{&tSF%hh4UBx52z@|;shR`xYl$tv0BJw?q`QJO%3Jr_$PMih9xQ^MY9Y!KQWR;Wp)#<)~I|x z{Asm`e8uYv8Jj%I169diuzU6;!6tssIJ0H2n6AhftN%u{C^P#*r{W7;*ihGAtwi8QsbB6nXdQ;Ht#mXF(Ah`%|4jH)}PeOFG;za;MX|UKajo{ z>F_}32s1qgM^s%XKfEv^coQFBU2}@0rH75hDxE?{ZZA29?umaH|70=x!?2m_9ZcDK zO+VNi#dzQJ#jCz5lJ{=^$+1nB)NtAi%*s%u(oaQbW7Crmybt0v zf*0R%@SIdQQi6P-%=oj#tG_@tnBNoF70tz;W}-`$4oL(~1e9O|T`b1=NNhfMeOC1- ze#PiCaT{A~JBbrC4}}dnBX&vNv#yvXX-j7A`J@()qfl*T|9sAJJ~4pc*4S=r?6lml{A2V67T$h0dcMT9>9 z7#`aV*-Eyb;dozLjiRglqgZY#-6ITB7NFOs0T5xn^5nV~G(~gy>dk6ao7_+;&NxfC zqpwCFI{}ojy*&uFt!ZqDG|efeH2oqP$ZQRtRwAv(cjzDcbC#1u2n7R^JKH5_qn+XP z9`hGX)?^m^rxc_o_%+Qi;sP0fs{pWR!ivet9|JTE;O9u8IA~ecgS!B`m!JSL$BnRLYjiIIn@$?5x92F20zeB3Q3>lH VpZGYOx?0x(xQCZ}!@;m?{{{O1KS=-p literal 0 HcmV?d00001 diff --git a/content/blog/MOP2/fatfs-integration.adoc b/content/blog/MOP2/fatfs-integration.adoc new file mode 100644 index 0000000..9f26316 --- /dev/null +++ b/content/blog/MOP2/fatfs-integration.adoc @@ -0,0 +1,189 @@ += FAT filesystem integration into MOP2 +Kamil Kowalczyk +2025-11-19 +:jbake-type: post +:jbake-tags: MOP2 osdev +:jbake-status: published + +Hello, World! + +I would like to present to you FAT filesystem integration into the MOP2 operating system! This was +possible thanks to `fat_io_lib` library by UltraEmbedded, because there's no way I'm writing an +entire FAT driver from scratch ;). + +Source for `fat_io_lib`: https://github.com/ultraembedded/fat_io_lib + +== Needed changes and a "contextual" system + +Integrating fat_io_lib wasn't so straightforward as I thought it would be. To understand the problem +we need to first understand the current design of MOP2's VFS. + +=== MOP2's VFS explained + +The VFS (ie. Virtual File System) is a Windows/DOS-style labeled VFS. Mountpoints are identified by +a label, like for eg. `sys:/boot/mop2` or `base:/scripts/mount.tb`, which kinda looks like +`C:\Path\To\File` in Windows. I've chosen this style of VFS over the UNIX kind, because it makes more +sense to me personally. A mountpoint label can point to a physical device, a virtual device (ramsd) +or a subdevice/partition device. In the UNIX world on the other hand, we have a hierarchical VFS, so +paths look like `/`, `/proc`, `/dev`, etc. This is a little confusing to reason about, because you'd +think that if `/` is mounted let's say on a drive `sda0`, then `/home` would be a home directory +within the root of that device. This is not always the case. `/` can be placed on `sda0`, but `/home` +can be placed on `sdb0`, so now something that looks like a subdirectory, is now a pointer to an +entirely different media. Kinda confusing, eh? + +=== The problem + +So now that we know how MOP2's VFS works, let's get into low-level implementation details. To handle +mountpoints we use a basic hashtable. We just map the label to the proper underlying file system +driver like so: + +[source] +---- +base -> Little FS +uhome -> Little FS +sys -> FAT16 +---- + +Through this table, we can see that we have two mountpoints (base and uhome), which both use Little +FS, but are placed on different media entirely. Base is located on a ramsd, which is entirely virtual +and uhome is a partition on a physical drive. + +To manage such setup, we need to have separate filesystem library contexts for each mountpoint. +A context here means an object, which stores info like block size, media read/write/sync hooks, +media capacity and so on. Luckly, with Little FS we could do this out of the box, because it blesses +us with `lfs_t` - an instance object. We can then create this object for each Little FS mountpoint. + +.Internals of the VFS mountpoint structure +[source,c] +---- +typedef struct VfsMountPoint { + int _hshtbstate; + char label[VFS_MOUNTPOINT_LABEL_MAX]; + + int32_t fstype; + StoreDev *backingsd; + + VfsObj *(*open)(struct VfsMountPoint *vmp, const char *path, uint32_t flags); + int32_t (*cleanup)(struct VfsMountPoint *vmp); + int32_t (*stat)(struct VfsMountPoint *vmp, const char *path, FsStat *statbuf); + int32_t (*fetchdirent)(struct VfsMountPoint *vmp, const char *path, FsDirent *direntbuf, size_t idx); + int32_t (*mkdir)(struct VfsMountPoint *vmp, const char *path); + int32_t (*delete)(struct VfsMountPoint *vmp, const char *path); + + // HERE: instance objects for the underlying filesystem driver library. + union { + LittleFs littlefs; + FatFs fatfs; + } fs; + SpinLock spinlock; +} VfsMountPoint; + +typedef struct { + SpinLock spinlock; + VfsMountPoint mountpoints[VFS_MOUNTPOINTS_MAX]; +} VfsTable; + +// ... +---- + +.Little FS init code +[source,c] +---- +int32_t vfs_init_littlefs(VfsMountPoint *mp, bool format) { + // Configure Little FS + struct lfs_config *cfg = dlmalloc(sizeof(*cfg)); + memset(cfg, 0, sizeof(*cfg)); + cfg->context = mp; + cfg->read = &portlfs_read; // Our read/write hooks + cfg->prog = &portlfs_prog; + cfg->erase = &portlfs_erase; + cfg->sync = &portlfs_sync; + cfg->block_size = LITTLEFS_BLOCK_SIZE; + cfg->block_count = mp->backingsd->capacity(mp->backingsd) / LITTLEFS_BLOCK_SIZE; + // left out... + int err = lfs_mount(&mp->fs.littlefs.instance, cfg); + if (err < 0) { + ERR("vfs", "Little FS mount failed %d\n", err); + return E_MOUNTERR; + } + + // VFS hooks + mp->cleanup = &littlefs_cleanup; + mp->open = &littlefs_open; + mp->stat = &littlefs_stat; + mp->fetchdirent = &littlefs_fetchdirent; + mp->mkdir = &littlefs_mkdir; + mp->delete = &littlefs_delete; + return E_OK; +} +---- + +So where is the problem? It's in the fat_io_lib library. You see, the creators of Little FS thought +this out pretty well and designed their library in such manner that we can do cool stuff like this. +The creators of fat_io_lib on the other hand... yeah. Here are the bits of internals of fat_io_lib: + +[source,c] +---- +//----------------------------------------------------------------------------- +// Locals +//----------------------------------------------------------------------------- +static FL_FILE _files[FATFS_MAX_OPEN_FILES]; +static int _filelib_init = 0; +static int _filelib_valid = 0; +static struct fatfs _fs; +static struct fat_list _open_file_list; +static struct fat_list _free_file_list; +---- + +There's no concept of a "context" - everything is thrown into a bunch of global variables. +To clarify: THIS IS NOT BAD! This is good if you need a FAT library for let's say a microcontroller, +or some other embedded device. Less code/stuff == less memory usage and so on. + +When I was searching online for a FAT library, I wanted something like Little FS, but to my suprise +there are no libraries for FAT designed like this? Unless it's a homebrew OsDev project, of course. +I've even went on reddit to ask and got nothing, but cricket noises ;(. + +I've decided to modify fat_io_lib to suit my needs. I mean, most of the code is already written for +me anyway, so I'm good. + +The refactoring was quite easy to my suprise! The state of the library is global, but it's all nicely +placed in one spot, so we can then move it out into a struct: + +[source,c] +---- +struct fat_ctx { + FL_FILE _files[FATFS_MAX_OPEN_FILES]; + int _filelib_init; + int _filelib_valid; + struct fatfs _fs; + struct fat_list _open_file_list; + struct fat_list _free_file_list; + void *extra; // we need this to store a ref to the mountpoint to access storage device hooks +}; +---- + +[source,c] +---- +// This is what our VfsMountPoint struct from earlier was referencing, BTW. +typedef struct { + struct fat_ctx instance; +} FatFs; +---- + +I've then gone on a compiler error hunt for about 2 hours! All I had to do is change references for +eg. from `_fs.some_field` into `ctx->_fs.some_field`. It was all pretty much brainless work - just +compile the code, read the error line number, edit the variable reference, repeat. + +== Why do I even need FAT in MOP2? + +I need it, because Limine (the bootloader) uses FAT16/32 (depending on what the user picks) to store +the kernel image, resource files and the bootloader binary itself. It'd be nice to be able to view +all of these files and manage them, to maybe in the future for eg. update the kernel image from the +system itself (self-hosting, hello?). + +== Fruits of labour + +Here are some screenshots ;). + +image::/img/fatfs-showcase2.png["showcase 2"] +image::/img/fatfs-showcase1.png["showcase 1"]