SdFat with support for long filenames

WereCatf
Sun Apr 24, 2016 8:28 am
Greiman added support for STM32F1 in his still-in-beta version of SdFat at https://github.com/greiman/SdFat-beta

The library supports long filenames and all such fancy stuffs, and uses DMA to speed up transfers and supports SPI-transactions.


martinayotte
Sun Apr 24, 2016 12:34 pm
Yes, I’m using the Greiman SdFat for almost a year, a snapshot from his github done before the Beta version, so I had to tweak it a bit.

Pito
Mon Apr 25, 2016 5:38 pm
I’ve tried Bill’s SdFat-beta on Maple Mini. Works fine on SPI2 and PA8 /CS. None tweaking, only edited the chipselect pin and added SPI.setModule(2); before sdcard’s begin() (as per Bill’s hint).
Not sure about the bench write speed, however. A junk box sdcard.
PS:
It seems there is an issue with SPI speed setting as the writes are quite slow :( ..
So a tweak is needed..

Type any character to start

This program can erase and/or format SD/SDHC cards.

Erase uses the card's fast flash erase command.
Flash erase sets all data to 0X00 for most cards
and 0XFF for a few vendor's cards.

Cards larger than 2 GB will be formatted FAT32 and
smaller cards will be formatted FAT16.

Warning, all data on the card will be erased.
Enter 'Y' to continue: Y

Options are:
E - erase the card and skip formatting.
F - erase and then format the card. (recommended)
Q - quick format the card without erase.

Enter option: F
Card Size: 3951 MB, (MB = 1,000,000 bytes)

Erasing
..............................
All data set to 0x00
Erase done

Formatting
Blocks/Cluster: 64
FAT32
........
Format done
Type any character to start
FreeStack: 14871
Type is FAT32
Card size: 3.95 GB (GB = 1E9 bytes)

Manufacturer ID: 0X2
OEM ID: TM
Product: SA04G
Version: 0.6
Serial number: 0X242C2445
Manufacturing date: 6/2011

File size 5 MB
Buffer size 512 bytes
Starting write test, please wait.

write speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
23.45,645886,2269,21816
23.45,497538,2270,21819

Starting read test, please wait.

read speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
404.73,101411,926,1263
407.97,1634,926,1253

Done
Type any character to start

SdFat version: 20160422

Assuming the SD is the only SPI device.
Edit DISABLE_CHIP_SELECT to disable another device.

Assuming the SD chip select pin is: 27
Edit SD_CHIP_SELECT to change the SD chip select pin.

type any character to start

init time: 2 ms

Card type: SDHC

Manufacturer ID: 0X2
OEM ID: TM
Product: SA04G
Version: 0.6
Serial number: 0X242C2445
Manufacturing date: 6/2011

cardSize: 3951.03 MB (MB = 1,000,000 bytes)
flashEraseSize: 128 blocks
eraseSingleBlock: true
OCR: 0XC0FF8000

SD Partition Table
part,boot,type,start,length
1,0X0,0XB,8192,7708672
2,0X0,0X0,0,0
3,0X0,0X0,0,0
4,0X0,0X0,0,0

Volume is FAT32
blocksPerCluster: 64
clusterCount: 120320
freeClusters: 120166
freeSpace: 3937.60 MB (MB = 1,000,000 bytes)
fatStartBlock: 14502
fatCount: 2
blocksPerFat: 941
rootDirStart: 2
dataStartBlock: 16384

type any character to start


Pito
Wed Apr 27, 2016 4:00 pm
“bench” with Buffer size 8192, the same sdcard as above.
=========================

File size 5 MB
Buffer size 8192 bytes
Starting write test, please wait.

write speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
1035.03,564362,6300,7905
1061.86,388000,6299,7705
1063.90,375521,6301,7690
1044.55,373082,6297,7832
1052.91,368699,6301,7706
1011.97,361148,6300,7871

Starting read test, please wait.

read speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
1140.63,8157,5651,7181
1140.90,8148,5653,7179
1140.37,8147,5652,7181
1140.37,8157,5661,7181
1140.63,8151,5654,7181
1140.63,8154,5651,7181

Done
Type any character to start


WereCatf
Wed Apr 27, 2016 4:15 pm
Pito wrote:“bench” with Buffer size 8192, the same sdcard as above.

WereCatf
Wed Apr 27, 2016 4:19 pm
Pito wrote:It seems there is an issue with SPI speed setting as the writes are quite slow :( ..

Pito
Wed Apr 27, 2016 6:31 pm
Bill has done a fix today as the DMA/noneDMA switch was broken a little bit.
The above today’s results are with SPI1 and the divider is DIV2. The card is an old Kingston microsd 4GB class4.
With buffer size 512 in the “bench” it does not matter what the SPI speed is (DIV2 DIv4 DIV8 DIV16), I always get around 23kB/sec write speed. The read scales with the SPI speed, however.
That is similar to the SDIO performance I tested 4y back on stm32f407discovery – the block size 512bytes did ~300kB/s write, in order to get ie 4-5MB/sec write speed I had to use 16kB or 32KB large block size.
PS: as we did experiments with Bill today his Kingston of the same params as mine but 2y older did ~70kB/sec write with buffer size 512b in bench. With 8kB buffer his card did 1.4MB/sec. So the older the card the faster :)

WereCatf
Wed Apr 27, 2016 8:19 pm
Pito wrote:So the older the card the faster :)

Pito
Thu Apr 28, 2016 6:30 am
The sdcards are all different. The fastest card I saw are those 64-128MB one. The newer cards do a lot more housekeeping (like wear leveling and others), and, they use much bigger flash erase blocks, so the overhead with processing smaller data volumes might be bigger.
Also there is a difference in strategies the sdcards use. There are cards which perform nice with filesystems (ie Samsung) and others which do better with writing large chunks. We messed a lot with such measurements at retrobsd.org in past – see for example the results
https://github.com/sergev/LiteBSD/wiki/ … -Benchmark
http://retrobsd.org/wiki/doku.php/doc/sd-benchmark

Pito
Thu Apr 28, 2016 8:39 pm
Sandisk 16GB CL10 UHS-I
=========================
File size 5 MB
Buffer size 8192 bytes
Starting write test, please wait.

write speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
834.80,619179,7542,9803
828.85,621921,7736,9873
752.01,621304,7740,10883
830.36,620154,7733,9856
753.71,617407,7738,10858
830.91,617178,7735,9847

Starting read test, please wait.

read speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
1445.09,6437,5649,5667
1445.93,6434,5646,5665
1445.09,6445,5649,5667
1444.67,6429,5653,5667
1445.09,6436,5649,5667
1445.09,6435,5653,5667

Samsung Evo 16GB UHS-I CL10
===========================
File size 5 MB
Buffer size 8192 bytes
Starting write test, please wait.

write speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
1202.68,15608,5888,6803
1210.84,21699,5885,6756
1206.16,23213,5888,6782
1204.71,23194,5888,6792
1211.13,21612,5888,6756
1206.45,23179,5889,6782

Starting read test, please wait.

read speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
1427.34,7057,5670,5737
1427.75,7053,5664,5735
1428.16,7049,5671,5735
1427.75,7051,5670,5736
1428.16,7041,5668,5735
1427.75,7052,5668,5736


stevech
Sun May 01, 2016 3:16 am
Pito wrote:The sdcards are all different. The fastest card I saw are those 64-128MB one. The newer cards do a lot more housekeeping (like wear leveling and others), and, they use much bigger flash erase blocks, so the overhead with processing smaller data volumes might be bigger.
Also there is a difference in strategies the sdcards use. There are cards which perform nice with filesystems (ie Samsung) and others which do better with writing large chunks. We messed a lot with such measurements at retrobsd.org in past – see for example the results
https://github.com/sergev/LiteBSD/wiki/ … -Benchmark
http://retrobsd.org/wiki/doku.php/doc/sd-benchmark

Pito
Sun May 01, 2016 7:39 am
Yea, the CRC was set to 2, the slowest, so this is with CRC off.
Mind the Sandisk is most probably a fake or defect, as the max write latency increases over time from 600ms to 1400ms :twisted:
Samsung Evo 16GB CL10 UHS-I
=========================
File size 5 MB
Buffer size 8192 bytes
Starting write test, please wait.

write speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
1737.52,96531,2312,4701
1950.48,91484,2313,4165
1717.22,92899,2313,4759
1787.24,92709,2313,4572
1798.82,135607,2313,4543
1772.66,92645,2313,4610

read speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
2929.14,4870,2739,2794
2932.58,3622,2740,2790
2934.30,3679,2737,2791
2934.30,3641,2738,2791
2934.30,3624,2738,2790
2934.30,3624,2738,2791

Sandisk 16GB CL10 UHS-I
========================
File size 5 MB
Buffer size 8192 bytes
Starting write test, please wait.

write speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
865.30,1406436,4460,9458
883.98,1404723,4457,9256
708.01,1410744,4401,11559
855.67,1403142,4400,9562
708.61,1406538,4458,11550
881.02,1417802,4401,9287

read speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
2901.93,5164,2756,2821
2908.68,3474,2756,2815
2905.30,3475,2756,2817
2905.30,3478,2757,2818
2905.30,3479,2757,2818
2905.30,3477,2758,2818


Pito
Fri May 06, 2016 1:15 pm
A second Sandisk Ultra 16GB CL10 UHS-I, MMini, SPI1 @36MHz, SdFat “bench”, seems it is not a fake, but write latency is high, indeed (not freshly formatted):
Type any character to start
FreeStack: 7191
Type is FAT32
Card size: 15.93 GB (GB = 1E9 bytes)

Manufacturer ID: 0X3
OEM ID: SD
Product: SL16G
Version: 8.0
Serial number: 0X424D3885
Manufacturing date: 8/2015

File size 5 MB
Buffer size 8192 bytes
Starting write test, please wait.

write speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
1792.37,706978,2555,4565
1689.36,733356,2551,4843
1693.94,711411,2542,4831
1575.88,712173,2535,5190
1689.36,708789,2548,4843
1685.37,708251,2556,4855

Starting read test, please wait.

read speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
2985.14,3259,2679,2743
2983.36,3129,2686,2744
2983.36,3138,2681,2744
2983.36,3129,2680,2744
2983.36,3134,2683,2744
2983.36,3133,2680,2744

Done
Type any character to start


RogerClark
Fri May 06, 2016 9:39 pm
Perhaps some cards automatically switch to the high speed mode if they detect a high SPI clock speed

Leave a Reply

Your email address will not be published. Required fields are marked *