Spark Logic

New SdFat library (with and without DMA)

victor_pv Mon Apr 27, 2015 7:11 pm

***Latest Update 04/10/2017***
The code in the official repo from Bill Greyman has now the support for the libmaple based core (stm32duino) integrated in it. So no need to download it from my repo any more.

Official SD-Fat library repository:
https://github.com/greiman/SdFat

===============================================================

UPDATE TO THIS WARNING. SHOULD BE SOLVED IN THE LATEST REPO. If you have problems using pin 7 for CS, make sure you are in sync with Roger’s master STM32Duino repo.
WARNING!!
I was just testing an SD card with one of the sketches, and noticed that you CAN NOT use pin 7 in the maple mini (the stm32f103 native SS pin for spi1) for CS. I have not looked at how the library manages CS and how that pin is setup in out SPI library, but I guess there is a conflict in the pin mode. So the work around at the moment is to use a different pin for CS.
– Updated 06/04/2015-

– New update on 07/10/2015 – Small code changes to the library and specially to the SPI library, which should go to Roger’s master repo soon.

So as I posted before in the forum, I modified the latest SdFat library from Greiman.
I had submitted him changes for a previous version, but had issues. Now all the issues seem to be corrected in my latest version.
It is available here:
https://github.com/victorpv/SdFat

It seems to work fine for me for the few examples I have tested.
If you want the best performance for large transfers, can use DMA mode.

DMA would be advantageous for large transfers, but doesn’t help at all, and may even have a penalty, if you are just reading a few bytes to a log and things like that. Now if you transfer images or mp3 files from the sdcard, DMA can improve performance greatly, anything involving big blocks at once.

DMA can be enabled or disabled editing SdSpiSTM32F1.cpp, there is one #define line:
/** Use STM32 DMAC if nonzero */
#define USE_STM32F1_DMAC 1

Comment that line or change the 1 for 0, and DMA transfers will not be used.

RECOMMENDATIONS FOR SPEED:
Use DMA mode.
Set a bigger buffer, ideally 4KB

Read speeds without DMA, 4KB buffer:
read speed and latency
speed,max,min,avg
KB/Sec,usec,usec,usec
676.29,8189,5927,6056
676.38,7467,5927,6054
676.38,7468,5927,6054
676.38,7469,5926,6054
676.38,7472,5926,6054

Replies

bubulindo Fri Sep 18, 2015 7:55 am

Ups… found it.

Although, it is a bit clunky having to dig through the library files to find which SPI was configured. Since the CS pin is possible to configure, it would make sense to be able to configure which SPI peripheral to use. But… that’s just my opinion.

victor_pv Fri Sep 18, 2015 11:42 am

bubulindo wrote:Ups… found it.

Although, it is a bit clunky having to dig through the library files to find which SPI was configured. Since the CS pin is possible to configure, it would make sense to be able to configure which SPI peripheral to use. But… that’s just my opinion.

bubulindo Fri Sep 18, 2015 5:13 pm

Hello,

In my case, I need to change to SPI2 as my board has the SD card wired into SPI2 (nothing I can do about it)… considering that another user has it on SPI3, it’s already two people that had to change the library because it’s not “possible” to use SPI1.

I remember testing an SdFat library about two years ago and we passed an SPI object for it to run the SD functions on. It was just a little bit more transparent, I guess.

Still, thanks. I’ve tested one of the examples and it recognized the card.

If I wanted to log AD values into the SD, what would be my maximum achievable frequency? Any numbers off the top of your heads?

stevech Fri Sep 18, 2015 6:21 pm

PJRC’s open source “SPI transactions” allows for coordinated sharing of an SPI port among multiple slave devices.
It does NOT though allow for a real time device like an always-on telemetry radio using SPI – because it needs to have the RX on all the time and that can create an interrupt at any time, and the ISR uses the SPI.

For SD cards, etc, SPI transactions is fine

e-dredon Mon Sep 28, 2015 10:51 am

Good afternoon all.

I didn’t have any success yet with the SD module of my ILI9341 LCD display, and the optimized SdFat library.

To be sure It’s not an hardware issue with my ILI9341, I’ve tried with a Teensy 3.1.
Paul PaulStoffregen spent some time doing similar improvements to the teensy 3.xx Sd library.
I can get read/write working only when I disable the optimization flag in the library configuration file (optimization flag doesn’t work with any of my microSD (512MB, 2GB class 4, 8GB class 4, 16GB class 12), or should it be because the module doesn’t have the SPI RAM populated on the board.
Here is a link on our talk on the PJRC forum :
https://forum.pjrc.com/threads/29331-TF … #post81454

Now with my STM32F103C8T6 (bluepill)
I tried EVERY digital pins for the SD_CS line.
I’ve disabled the DMA optimization for initial tests in the config file SdSpiSTM32F1.cpp
#define USE_STM32F1_DMAC 0

I’ve tried to use SPI1 for both the ILI9341 display and SD module, with a dedicated CS pin for the SD_CS.
I’ve tried the latest SPI library with the new SPI.setModule(2); in the setup section of my code. In that case I hooked the SD miso/mosi/sck on the SPI2, and specified the SD_CS (I tried the SS of the SPI2 and other pins like 0, 1 because I’ve seen some users got success with a mapple mini with that configuration)
Last but not least, I’ve tried various DIV_CLOCK settings (2, 4, 8) with no success too.
const uint8_t spiSpeed = SPI_CLOCK_DIV8;

If I define a pin with the STM32F103C8 naming scheme (PAxx/PBxx/PCxx, etc…), the display works, but, if i compile the sdinfo test sketch, I have a different numbering on the serial monitoring. For exemple, my PB13 has the pin number 33 shown on the serial monitor.

I don’t know what else to do now…

here is the code i use:

#include <SPI.h>
#include <SdFat.h>
#include <SdFatUtil.h>
#include <ILI_SdSpi.h>
#include <ILI_SdFatConfig.h>
#include <ILI9341_due_gText.h>
#include <ILI9341_due.h>

// CS and DC for the LCD
#define LCD_DC PA1
#define LCD_CS PA4
#define LCD_RST PA0

#define SD_CS PB12 // Chip Select for SD card

#define BUFFPIXELCOUNT 320 // size of the buffer in pixels
#define SD_SPI_SPEED SPI_HALF_SPEED // SD card SPI speed, try SPI_FULL_SPEED
const uint8_t spiSpeed = SPI_CLOCK_DIV8;
SdFat sd; // set filesystem
SdFile bmpFile; // set filesystem
//ArduinoOutStream cout(Serial);

ILI9341_due tft(LCD_CS, LCD_DC, LCD_RST);

// store error strings in flash to save RAM
#define error(s) sd.errorHalt_P(PSTR(s))

void setup()
{
Serial.begin(9600);

SPI.setModule(2);
tft.begin();
tft.setRotation(iliRotation270); // landscape
progmemPrint(PSTR("Initializing SD card..."));

if (!sd.begin(SD_CS, SD_SPI_SPEED)){
progmemPrintln(PSTR("failed!"));
return;
}
progmemPrintln(PSTR("OK!"));
}

stevestrong Mon Oct 19, 2015 8:43 pm

Hello all,
I have a problem with my SD card connected to a Maple mini clone.
I run the SdInfo sketch combined with the RawWrite sketch and it seems that no data can be written to the SD card.
Installed the latest versions from victorpv/SdFat and from the rogerclarkmelbourne/Arduino_STM32 github.
I am using SPI 2 with SPI_CLOCK_DIV8.
Reading from the SD card works. The card was formatted with SdFormatter.
A serial log looks like this:
SdFat version: 20150321

Assuming the SD is the only SPI device.
Edit DISABLE_CHIP_SELECT to disable another device.

Assuming the SD chip select pin is: 31
Edit SD_CHIP_SELECT to change the SD chip select pin.

type any character to start

init time: 1 ms

Card type: SD2

Manufacturer ID: 2
OEM ID: TM
Product: SA02G
Version: 0.3
Serial number: A07C6B9C
Manufacturing date: 12/2008

cardSize: 1973.42 MB (MB = 1,000,000 bytes)
flashEraseSize: 128 blocks
eraseSingleBlock: true
OCR: 80FF8000

SD Partition Table
part,boot,type,start,length
1,0,6,135,3854201
2,0,0,0,0
3,0,0,0,0
4,0,0,0,0

Volume is FAT16
blocksPerCluster: 64
clusterCount: 60214
freeClusters: 60212
freeSpace: 1973.03 MB (MB = 1,000,000 bytes)
fatStartBlock: 136
fatCount: 2
blocksPerFat: 236
rootDirStart: 608
dataStartBlock: 640

Starting RawWrite...
removing RAW.TXT...
creating contiguous...
createContiguous failed
SD errorCode: 13
SD errorData: FF

RogerClark Mon Oct 19, 2015 9:04 pm

Welcome Steve…

Did you try on SPI1

SPI.setModule() is a fairly new feature, and I can’t recall if anyone fully tested it with SD cards

stevestrong Mon Oct 19, 2015 9:28 pm

I just finished to try with SPI 1, first with SPI.setModule(1); and second with this line commented out, in both cases keeping the same SS line (31).
In both cases I get cardBegin error 0x02 0xFF.
What am I doing wrong?

madias Mon Oct 19, 2015 9:42 pm

I only got the new SdFat library running with a specific card holder (without voltage converters, even useless for our 3.3V boards…) and specific SD-cards, does it work with the “standard” SD library?

stevestrong Mon Oct 19, 2015 9:58 pm

Well, I haven’t tried with the standard lib.
But I have some news.
I tried to visualize the SPI CLK signal on my oscilloscope and – voila – it started to work on SPI 1!
But not on SPI 2.
It seems that the impedance on the SCK line is critical.

Can any of you acknowledge this?

RogerClark Mon Oct 19, 2015 10:18 pm

Steve

You don’t need to call setModule(1) for SPI 1, its only required if you need to move the SPI class to use SPI2 or on some boards SPI3

Re: SCLK

I’ve not tried this on the Maple mini

I did build a simple audio player that read wav’s from a SD card on SPI1 and played them out a SPI audio interface on SPI2 and that worked reasonably well, but I did get occasional gitches on reading the SD card.

But the original SD card lib that the Arduino uses was not that good, and I recall that there has been some work done recently to improve it.

I think @victorPV may be the person who knows the most about this, as he wrote the DMA support for SPI and had it working with SD cards at one point.

stevestrong Mon Oct 19, 2015 10:26 pm

Roger, thanks, I’m going to spend more time analyzing the signals with the scope, and compare SPI 1 to SPI 2 signals.
For SPI 1, it’s true, setModule(1) has no influence.
I’ll post later some results.

victor_pv Mon Oct 19, 2015 10:52 pm

stevestrong wrote:Hello all,
I have a problem with my SD card connected to a Maple mini clone.
I run the SdInfo sketch combined with the RawWrite sketch and it seems that no data can be written to the SD card.
Installed the latest versions from victorpv/SdFat and from the rogerclarkmelbourne/Arduino_STM32 github.
I am using SPI 2 with SPI_CLOCK_DIV8.
Reading from the SD card works. The card was formatted with SdFormatter.
A serial log looks like this:
SdFat version: 20150321

Assuming the SD is the only SPI device.
Edit DISABLE_CHIP_SELECT to disable another device.

Assuming the SD chip select pin is: 31
Edit SD_CHIP_SELECT to change the SD chip select pin.

type any character to start

init time: 1 ms

Card type: SD2

Manufacturer ID: 2
OEM ID: TM
Product: SA02G
Version: 0.3
Serial number: A07C6B9C
Manufacturing date: 12/2008

cardSize: 1973.42 MB (MB = 1,000,000 bytes)
flashEraseSize: 128 blocks
eraseSingleBlock: true
OCR: 80FF8000

SD Partition Table
part,boot,type,start,length
1,0,6,135,3854201
2,0,0,0,0
3,0,0,0,0
4,0,0,0,0

Volume is FAT16
blocksPerCluster: 64
clusterCount: 60214
freeClusters: 60212
freeSpace: 1973.03 MB (MB = 1,000,000 bytes)
fatStartBlock: 136
fatCount: 2
blocksPerFat: 236
rootDirStart: 608
dataStartBlock: 640

Starting RawWrite...
removing RAW.TXT...
creating contiguous...
createContiguous failed
SD errorCode: 13
SD errorData: FF

RogerClark Mon Oct 19, 2015 11:24 pm

The uno will feed 5v, so all is fine, but you may be feeding 3.3V form the mini to a regulator on board the sdcard module…

On the Maple mini, I think you can get the incoming 5V from the USB input via the VIN pin. But I can’t remember if its via a diode from USB power, and when I tested on one of my Maple minis it showed 4.8V

Just check whatever you feed your SD card module with, that it does seem to give 3.3V to the SD card, as I’ve noticed that some USB outputs are as low as 4.75V and sometimes the regulators won’t work unless you supply around 5V

stevestrong Tue Oct 20, 2015 11:12 am

The maple mini clone (Baite) is fed from USB 5V and it has its own 3.3V regulator (LM1117-3.3V) on board. All components are fed with this 3.3V.
The SD card module has no regulator, and gets the 3.3V from the mini board. So no level converters are involved.
The card has class 10.
But I find strange that in order to work, the SCK line (SPI 1) had to be connected to the scope probe.
What does it actually mean? Wrong impedance?
Are the wires from Maple mini board to SD card module too long? Or too short?
Of course, the shorter the better, one would say. But the same length had no effect on Pro Mini with 8MHz SPI clock, fed from 3.3V supplied by a USB-serial converter board.
Maple mini SPI clock is 12 MHz according to the scope (DIV4). Is this 4MHz more than Pro Mini making so huge issues? Or is rather the shape of the signal?
Or is CPOL and/or CPHA issue? Or maybe NSS pin management related?
This is the direction I want to make more scope recordings, I think this would give answer to some users here in forum.
What do you think?
Does it make sense?
Meanwhile I will try to put an extra 10uF filtering capacitor to the 3.3V line and check whether it makes a difference or not.

Oh yes, I would like to finally have the SD card working on SPI 2 since I need the analog input pins 0…9 for my project.

ahull Tue Oct 20, 2015 3:11 pm

stevestrong wrote:The maple mini clone (Baite) is fed from USB 5V and it has its own 3.3V regulator (LM1117-3.3V) on board. All components are fed with this 3.3V.
The SD card module has no regulator, and gets the 3.3V from the mini board. So no level converters are involved.
The card has class 10.
But I find strange that in order to work, the SCK line (SPI 1) had to be connected to the scope probe.
What does it actually mean? Wrong impedance?
Are the wires from Maple mini board to SD card module too long? Or too short?
Of course, the shorter the better, one would say. But the same length had no effect on Pro Mini with 8MHz SPI clock, fed from 3.3V supplied by a USB-serial converter board.
Maple mini SPI clock is 12 MHz according to the scope (DIV4). Is this 4MHz more than Pro Mini making so huge issues? Or is rather the shape of the signal?
Or is CPOL and/or CPHA issue? Or maybe NSS pin management related?
This is the direction I want to make more scope recordings, I think this would give answer to some users here in forum.
What do you think?
Does it make sense?
Meanwhile I will try to put an extra 10uF filtering capacitor to the 3.3V line and check whether it makes a difference or not.

Oh yes, I would like to finally have the SD card working on SPI 2 since I need the analog input pins 0…9 for my project.

stevestrong Tue Oct 20, 2015 3:21 pm

Andy, I will try resistor and then capacitor as load in parallel.
I rather think that the probe actually introduces flatness (filtering instead of inducing noise) and/or phase shift to the signal.
The SCK signal on the scope display was quite ok, no spikes, sinusoidal shape.
I will post some more results later.

zmemw16 Tue Oct 20, 2015 3:35 pm

isn’t the scope probe just slowing the rise time?
stephen

RogerClark Tue Oct 20, 2015 8:01 pm

What length of wire are you using between the Maple mini and the SD module.

Its always amazed me that running clocks and data at 10s of megahertz , down bits or normal wire or ribbon cable, works at all ;-)

These sorts of frequencies would have been considered “High Frequency” RF about 30 years ago.

To get sharp edges on square waves actually requires bandwidths of many hundred of Mhz.

Anyway.. I have experienced issues with crosstalk on ribbon cables on SPI, in the past,

So i suspect you are experiencing something similar, or ringing of the SPI clock or data lines.

stevestrong Tue Oct 20, 2015 10:25 pm

Here are my first recordings.

The sketch is started, and the recording is triggered by the very first rising edge of the SPI 1 clock SCK.
Yellow is the SPI 1 SCK (pin 6), blue is CS chip select (pin 31 in my case, which belongs to CS of SPI 2).

A successful start – only the first millisecond – I suppose this is the introducing part with “sd.cardBegin()”:

Image

red is the first zoom in:

Image

yes, clock pulses are activated before an active chip select!
and yes, the first 4 (as well as following consecutive group of 3) rising edges are slow.
and here the second zoom in (green rectangular form the first image):

Image

There are 2 kind of pulse trains: smaller (left side of the CS high period) and larger (right side of CS high period).
The smaller are (let’s call them) “high-frequency” pulses (27MHz), here the last train generated after CS goes “high” !!!

Image

and the larger are low-frequency (9MHz) pulses:

Image

I think, these are the right pulses, how they always should be. But they aren’t, specially not at the beginning, where 27MHz pulses are generated.
Then after some condition, the CS is generated correctly before SCK (second falling CS edge), and the SCK pulses are also ok, the frequency goes down to 9MHz, which was actually selected by the sketch with SPI_CLOCK_DIV8:

Image

a better zoom in:

Image

And, at the end, after second rising CS edge, again, there are some additional clock pulses generated:

Image

Well, I don’t know what exactly is happening here, but it definitely strange.
Can someone explain these?

EDIT:
It looks like I managed to solve the probe issue, I soldered a 10uF capacitor directly onto the SD card module (Vcc-GND).
Now the sketch works on SPI 1 without the probe contact on SCK.
Probably the cross-talk has been decreased.
But the recorded signals still remained same as posted above.
TODO: record signals on SPI 2.

Sign in to reply