Before this patch, when calling lfs_mkdir or lfs_file_open with root
as the target, littlefs wouldn't find the path properly and happily
run into undefined behaviour.
The fix is to populate a directory entry for root in the lfs_dir_find
function. As an added plus, this allowed several special cases around
root to be completely dropped.
Regression after ChanFS update: Due to parameter changes in the f_mkfs
function, the option to use a separate block for MBR (FDISK) was turned
back on. This should be off as it conflicts with an explicit MBR when
using the MBRBlockDevice.
Rather than tracking all in-flight blocks blocks during a lookahead,
littlefs uses an ack scheme to mark the first allocated block that
hasn't reached the disk yet. littlefs assumes all blocks since the
last ack are bad or in-flight, and uses this to know when it's out
of storage.
However, these unacked allocations were still being populated in the
lookahead buffer. If the whole block device fits in the lookahead
buffer, _and_ littlefs managed to scan around the whole storage while
an unacked block was still in-flight, it would assume the block was
free and misallocate it.
The fix is to only fill the lookahead buffer up to the last ack.
The internal free structure was restructured to simplify the runtime
calculation of lookahead size.
deepikabhavnani did the hard work in tracking this issue down. Block
addresses are not cast to the correct type until after multiplying to
convert to byte addresses. This results in an overflow when the storage
is larger than 4 GB.
FATFilesystem declares sector count and size as uint32_t and block
device class arguments are addr and size which is uint64_t
While passing arguments to program/read/write API's of block device,
multiplication of uint32_t*uint32_t was not typecasted properly to
uint64_t which resulted in MSB truncation.
Eg. If block 0x800000 is accessed with block size 0x200, addr to be
passed (0x800000*0x200)0x100000000, but actual address passed was 0x0
which resulted in over-writting the root directory, and hence corrupted
filesystem
In the open call, the LFS_O_TRUNC flag was correctly zeroing the file, but
it wasn't actually writing the change out to disk. This went unnoticed because
in the cases where the truncate was followed by a file write, the
updated contents would be written out correctly.
Marking the file as dirty if the file isn't already truncated fixes the
problem with the least impact. Also added better test cases around
truncating files.
A COMMON folder allows code reuse across different test cases. This
avoids code duplication or code enterying the application space.
The COMMON folder is uppercase to match naming conventions in Mbed OS.
This was a small hole in the logic that handles initializing the
lookahead buffer. To imitate exhaustion (so the block allocator
will trigger a scan), the lookahead buffer is rewound a full
lookahead and set up to look like it is exhausted. However,
unlike normal allocation, this rewind was not kept aligned to
a multiple of the scan size, which is limited by both the
lookahead buffer and the total storage size.
This bug went unnoticed for so long because it only causes
problems when the block device is both:
1. Not aligned to the lookahead buffer (not a power of 2)
2. Smaller than the lookahead buffer
While this seems like a strange corner case for a block device,
this turned out to be very common for internal flash, especially
when a handleful of blocks are reserved for code.
This bug was a result of an annoying corner case around intermingling
signed and unsigned offsets. The boundary check that prevents seeking
a file to a position before the file was preventing valid seeks with
positive offsets.
This corner case is a bit more complicated than it looks because the
offset is signed, while the size of the file is unsigned. Simply
casting both to signed or unsigned offsets won't handle large files.
The API is as per posix standard, but does not provide stats for file/directory.
Stats buffer (block size, total block count, free block count) is filled for
entire mounted filesystem.
ExhaustibleBlockDevice.h
- Fix typos for consistent spelling.
ObservingBlockDevice.h
- Fix typos for consistent spelling.
ReadOnlyBlockDevice.h
- Fix typos for consistent spelling.
README.md
- Fix typos, mostly for branding.
DESIGN.md
- Make minor changes for consistent spelling and precise language.
SPEC.md
- Make minor changes for consistent spelling and precise language.
README.md
- Make minor changes for consistent spelling and precise language.
Not all devices have enough heap to fit a simulated heap block device,
however using a simulated heap block device is preferred if available
(reduced flash wear, faster testing).
Added MBED_TEST_SIM_BLOCKDEVICE for tests that only need a simulated
block device (wear_leveling + resilience), and added support for targets
that are known to have enough heap.
Unfortunately there are several issues colluding to force the timeout
this high.
1. The tests push littlefs to the limits of how many errors it can
handle before failing. As a side effect this causes a massive amount
of debug/warn/error logging about the situation.
2. The logging can't be turned off for specific tests. Note: This might
change with the introduction of test-configs.
3. Logging is fixed to a baud rate of 9600. Previous testing was carried
out with a baud rate of 115200, which is the reason for the original
timeout.
The index field of FATFS_DIR does not encapsulate all the context
required to reposition the directory traversal. ChaN provides
f_rewinddir() but no directory seek, so rewind if necessary then step
through until the desired index is reached.
When building with VisualGDB the compilation fails due to the lack of required include files Dir.h and File.h.
Compiler report:
1>------ Build started: Project: exported-mbed-STM32F429-nucleo, Configuration: Debug VisualGDB ------
1> FileSystem.cpp
1> mbed-os/features/filesystem/FileSystem.cpp: In instantiation of 'class Managed<mbed::File>':
1> mbed-os\features\filesystem\FileSystem.cpp(141,19): note : required from here
1>mbed-os\features\filesystem\FileSystem.cpp(130,7): error : invalid use of incomplete type 'class mbed::File'
1> class Managed : public F {
1> ^~~~~~~
1> mbed-os\features\filesystem\FileSystem.cpp(18):
1> mbed-os\features\filesystem\FileSystem.h(39,7): note : forward declaration of 'class mbed::File'
1> class File;
1> ^~~~
1> mbed-os/features/filesystem/FileSystem.cpp: In member function 'virtual int mbed::FileSystem::open(mbed::FileHandle**, const char*, int)':
1>mbed-os\features\filesystem\FileSystem.cpp(141,19): error : cannot convert 'Managed<mbed::File>*' to 'mbed::File*' in initialization
1> File *f = new Managed<File>;
1> ^~~~~~~~~~~~~
1>mbed-os\features\filesystem\FileSystem.cpp(142,16): error : invalid use of incomplete type 'class mbed::File'
1> int err = f->open(this, path, flags);
1> ^~
1> mbed-os\features\filesystem\FileSystem.cpp(18):
1> mbed-os\features\filesystem\FileSystem.h(39,7): note : forward declaration of 'class mbed::File'
1> class File;
1> ^~~~
1>mbed-os\features\filesystem\FileSystem.cpp(144,16): warning : possible problem detected in invocation of delete operator: [-Wdelete-incomplete]
1> delete f;
1> ^
1>mbed-os\features\filesystem\FileSystem.cpp(141,11): warning : 'f' has incomplete type
1> File *f = new Managed<File>;
1> ^
1> mbed-os\features\filesystem\FileSystem.cpp(18):
1> mbed-os\features\filesystem\FileSystem.h(39,7): note : forward declaration of 'class mbed::File'
1> class File;
1> ^~~~
1> mbed-os\features\filesystem\FileSystem.cpp(144,16): note : neither the destructor nor the class-specific operator delete will be called, even if they are declared when the class is defined
1> delete f;
1> ^
1>mbed-os\features\filesystem\FileSystem.cpp(148,13): error : cannot convert 'mbed::File*' to 'mbed::FileHandle*' in assignment
1> *file = f;
1> ^
1> mbed-os\features\filesystem\FileSystem.cpp(18):
1> mbed-os\features\filesystem\FileSystem.h(39,7): note : class type 'mbed::File' is incomplete
1> class File;
1> ^~~~
1> mbed-os/features/filesystem/FileSystem.cpp: In instantiation of 'class Managed<mbed::Dir>':
1> mbed-os\features\filesystem\FileSystem.cpp(153,18): note : required from here
1>mbed-os\features\filesystem\FileSystem.cpp(130,7): error : invalid use of incomplete type 'class mbed::Dir'
1> class Managed : public F {
1> ^~~~~~~
1> mbed-os\features\filesystem\FileSystem.cpp(18):
1> mbed-os\features\filesystem\FileSystem.h(38,7): note : forward declaration of 'class mbed::Dir'
1> class Dir;
1> ^~~
1> mbed-os/features/filesystem/FileSystem.cpp: In member function 'virtual int mbed::FileSystem::open(mbed::DirHandle**, const char*)':
1>mbed-os\features\filesystem\FileSystem.cpp(153,18): error : cannot convert 'Managed<mbed::Dir>*' to 'mbed::Dir*' in initialization
1> Dir *d = new Managed<Dir>;
1> ^~~~~~~~~~~~
1>mbed-os\features\filesystem\FileSystem.cpp(154,16): error : invalid use of incomplete type 'class mbed::Dir'
1> int err = d->open(this, path);
1> ^~
1> mbed-os\features\filesystem\FileSystem.cpp(18):
1> mbed-os\features\filesystem\FileSystem.h(38,7): note : forward declaration of 'class mbed::Dir'
1> class Dir;
1> ^~~
1>mbed-os\features\filesystem\FileSystem.cpp(156,16): warning : possible problem detected in invocation of delete operator: [-Wdelete-incomplete]
1> delete d;
1> ^
1>mbed-os\features\filesystem\FileSystem.cpp(153,10): warning : 'd' has incomplete type
1> Dir *d = new Managed<Dir>;
1> ^
1> mbed-os\features\filesystem\FileSystem.cpp(18):
1> mbed-os\features\filesystem\FileSystem.h(38,7): note : forward declaration of 'class mbed::Dir'
1> class Dir;
1> ^~~
1> mbed-os\features\filesystem\FileSystem.cpp(156,16): note : neither the destructor nor the class-specific operator delete will be called, even if they are declared when the class is defined
1> delete d;
1> ^
1>mbed-os\features\filesystem\FileSystem.cpp(160,12): error : cannot convert 'mbed::Dir*' to 'mbed::DirHandle*' in assignment
1> *dir = d;
1> ^
1> mbed-os\features\filesystem\FileSystem.cpp(18):
1> mbed-os\features\filesystem\FileSystem.h(38,7): note : class type 'mbed::Dir' is incomplete
1> class Dir;
1> ^~~
1> mbed-os/features/filesystem/FileSystem.cpp: In instantiation of 'int Managed<F>::close() [with F = mbed::Dir]':
1> mbed-os\features\filesystem\FileSystem.cpp(162,1): note : required from here
1>mbed-os\features\filesystem\FileSystem.cpp(133,27): error : incomplete type 'mbed::Dir' used in nested name specifier
1> int err = F::close();
1> ~~~~~~~~^~
1> mbed-os/features/filesystem/FileSystem.cpp: In instantiation of 'int Managed<F>::close() [with F = mbed::File]':
1> mbed-os\features\filesystem\FileSystem.cpp(162,1): note : required from here
1>mbed-os\features\filesystem\FileSystem.cpp(133,27): error : incomplete type 'mbed::File' used in nested name specifier
========== Build: 0 succeeded, 1 failed, 0 up-to-date, 0 skipped ==========
Some block devices (for example I2C EEPROM) can be erased at the byte
level. These aren't really block devices, but fall under the block
device API. For these devices, the fat filesystem needs to use a lower
bound on the block size. In this case we used 512 bytes is used since
it is already a standard.
This function provides a shortcut to reformatting a mounted filesystem.
Also, since this function is a virtual member of the FileSystem class,
the user does not need to know the underlying filesystem to reformat
the underlying storage.
This is only an issue when multiple fatfss are used simultaneously.
Repeated use of a single fatfs instance (even with different storages)
do not show this issue.
Full support requires path prefixes being applied for every function
that takes a path. Note: this is only required filesystems after the
first mounted filesystem. The first filesystem has no penalty.
Unlike the other disk_ioctl options, GET_SECTOR_SIZE is interpreted
as a 16-bit WORD, instead of a 32-bit DWORD. This caused an unaligned
access error on M0 platforms.
At some point the "mount" parameter for "f_mount" was name "force". This
led to a bit of confusion that ended with the default mount function
never calling block device init.
This is fine, since the block device can be manually initialized, but
a better user experience is where the filesystem initializes the block
device for the user.
This is backwards compatible due to the repeatability of the block
device init functions.
In class MBRBlockDevice the tole32 function had used union member
names u32 and u8. The introduction of REALTEK_RTL8195AM cauesd a
conflict with type names in basic_types given they're aliased as
macros to uint32_t and uint8_t respectively.
The old open/opendir functions did not provide a route for errors and
relied on implementations manually setting errno. Updated to return
errors directly.
Required for other representations of FileSystems, ie LocalFileSystem
Introduces FileSystemHandle for the same behaviour as FileHandle and
DirHandle.
Requires the following to hook into file/dir lookup:
```
int open(FileHandle **file, const char *filename, int flags)
int open(DirHandle **dir, const char *path)
```
This hook is provided by the FileSystem class, so requires no changes
from implementations.
This is necessary for support of block devices with >512 byte
blocks, such as most SPI flash parts.
- Enabled support of up to 4096 byte blocks
- Added support for heap-backed buffers using _FS_HEAPBUF
- Necessary to avoid stack overflows
- Avoids over-aggresive allocations of _MAX_SS
- Enabled _FS_TINY to further reduce memory footprint
- Haven't found a downside for this yet except for possible
thread contention
- Implicit MBR still allowed during mount
- maintains storage compatibility
- Not needed
- MBR utility is not exposed through the FAT filesystem,
so the only used partition was always the first. Omitting
the MBR is functionally equivalent
- Saves a few blocks on storage for MBR + offset
for FAT alignment
- Duplicated with MBRBlockDevice
- The implicit MBR actually prevents nesting a FAT filesystem
in the MBRBlockDevice
The standard is intentionally vague on if filesystems must
have '.' and '..' entries, allowing filesystems to omit this
concept completely:
http://pubs.opengroup.org/onlinepubs/9699919799/functions/readdir.html
However, the '.' and '..' entries are common on FAT filesystems
and in most other filesystems.
This enables '.' and '..' entries in the FAT filesystem.
Should follow same path as FileHandle, although this is less used
and there is currently no route to introduce a hook for a customized
DirHandle in retarget.
As identified by @hasnainvirk, @kjbracey-arm, the FileHandle and
FileBase serve two separate functions and their integration is
limiting for certain use cases.
FileLike is actually the redundant class here, but the multiple
inheritance it provides is used as a hack by the retargeting code
to get at the FileHandle implementation bound to the FileBase name.
It may make more sense for the FileBase to inherit from FileHandle,
(with perhaps a different name), but rather than explore the
possibility, this will just restore the previous hierarchy.
Christopher Haster
Brendan McDonnell
Cruz Monrreal
Amanda Butler
Martin Kojtal
deepikabhavnani
Colin Hogben
Jimmy Brisson
Kevin Gilbert
Kevin Gilbert
0x6d61726b
Sam Grove
Kevin Bracey