#17 Hard Crash when using multiple files @ once.

open
nobody
None
5
2008-06-23
2008-06-23
Leeds
No

I have been trying to get EFSL to store log data on an SD card. In doing so I have found some pretty odd problems which I hope perhaps someone else here has encountered before.

I am using version 2.8 but I have also added each and every patch and bug-fix that has been posted to this board. I have had great success in using EFSL for small operations but now that I'm trying to use it more frequently for data logging I am finding that it is very, very prone to crashing.

Specifically- I am logging data on an sd card. Everything about these logs is currently a variable (the frequency of writing, the amount of data to write) but in general I'm logging data in 16 different files approximately once a minute.

If I open the file system and then one file, append to it, then close the file and close the filesystem all works great. I can do this around the clock it seems without any problems at all. The problem is that I'm spending enourmous amounts of processor time to log data this way, far more than I can allow.

When I try to open and append to each of the 16 files before closing the file system (open one, write to it, close it, then open the next file, etc) the task crashes and burns after about 100 or so writes.

I am only opening the file system from one task (from one function specifically) and I'm not doing anything crazy here, it just seems that there is sime bug that doesn't allow EFSL to properly open and close multiple files without reinitializing the entire filesystem.

My heap is huge and mostly unused as is my stack, yet something eventually kills the filesystem and leaves the card unreadable until I re-format it or run SCNDSK on the computer.

I've got the following settings:
#define IOMAN_NUMBUFFER 4
#define IOMAN_NUMITERATIONS 3
#define IOMAN_DO_MEMALLOC

#define CLUSTER_PREALLOC_FILE 1
#define CLUSTER_PREALLOC_DIRECTORY 0

Anyone else having these issues? It would be a real shame if 20 seconds out of every minute logged was used simply to store the data...

Discussion

  • damien h

    damien h - 2008-06-27

    Logged In: YES
    user_id=2129605
    Originator: NO

    Hi,

    well, I m more or less in the same case as you.

    I m using EFSL Lib 2.8 with a STR91x UP and FreeRTOS.
    I m logging: each second, data (2x 50 bytes) inside 2 different files, on a SD cards.

    After 4-5 hours of running (nothing is done during these 4-5 hours), my SD cards refuse some writting (number byte written not equal number byte should)...

    The step is done like this :
    1) Init of FileSystem.. If success, increase SPI Clk speed to 8Mhz

    for each second:
    {
    Open 1st file in append mode
    Write data in 1st file
    Close 1st file

    Open 2nd file in append mode
    Write data in 2nd file
    Close 2nd file

    if user ask to eject the card, release the File System
    }

    As you, I have huge stack as heap. My define is same as yours.

    I m little bit surprised of your: 20 sec to log data: seems you have a problem somewhere
    the only function which should take this time is to obtain free space (count free sector).

    Keep me in touch

    Regars.

    Damien

     
  • Nobody/Anonymous

    Logged In: NO

    I have run into the same problem as well, everything was working when I was mounting, opening, closing, unmounting, but then I decided to stop unmounting everytime and it started trashing my SD cards.

     
  • Nobody/Anonymous

    Possibly the problem could be down to the IOmanager not flushing the cache completely after a File close. If you open and close files enough times during that cycle it could overflow the cache, unless the cache manager implements a way of flushing the cache when its full.

    Anyway I forced a full flush on every fclose of the entire IO managers cache. It has little performance impact when not using multiple threads, and I will test it soon in a multythreaded environent.

    See my patch and fix here:
    http://sourceforge.net/tracker/index.php?func=detail&aid=2309775&group_id=131895&atid=722591

    James

     
  • James Walmsley

    James Walmsley - 2008-11-20

    Hi All,

    I've now tested this with my patch of flushing the entire cache on each fclose(). A quick preliminary result, I managed to open a file append to it, and then close it over 2000 times.

    I'm going to test this for longer. But first I shall remove my patch and see if I get a failure.

    Also to note, I've integrated efsl into our operating system, so the environment might be a little different from what you guys are doing.

    JAmes

     
  • James Walmsley

    James Walmsley - 2008-11-24

    I've been having some problems with this too, and after the patch that I mentioned below, it works fine. If the underlying SD-Card I/O driver is completely stable there shouldn't be any problems.

    So long as:

    All calls to the EFSL library are Atomic, i.e. A call to an LS function doesn't get called half way through a flush cache, or file open function call due to a context switch.

    To achieve this I wrapped all function calls to EFSL with a semaphore, to queue all use of the library. Its a little crude, and not ideal, but it functions great and prevents the EFSL cache being corrupted or out of context.

    I did have some problems using the SD Card with SPI on the BlackFIN 537 chips. This is because BlackFIN SPI in DMA doesn't stop the SPI clk after the transfer, and the SD Card went undefined. My work around this must have failed 1 in a few hundred thousand or so accesses to the SD.

    James

     

Log in to post a comment.