Ethernet library : fragmented sent packets

Hi,
I have a strange behaviour with a SEEED W5200 shield
I have an application using an Olimex STM32 board and a W5200 ethernet shield, where I communicate in TCP with an external system.
The application was written with the Maple IDE and works pretty well, except for a occasional communication errors that I have not investigated.
I am porting it to the Arduino IDE, and I have numerous errors that I have investigated.
What happens is that the external system expects TCP frames in character mode, and each command should be contained within a single frame to be correctly decoded on the external system. Examples :
A status request is “FPR”.
A command is “FPR remote=on”.
What happens is that very often (every other time !) the command is cut in two, for example :
“F” then “PR”
another example :
“FPR remot” then “e=on”.
I mean, the command is sent in two successive TCP frames (checked this with both the external application that protests about the truncated commands, and also by analyzing the traffic using Wireshark).
In both programming environments, the statement that sends the command is :
Client.println(“FPR remote=on”) ;
Strange ! Who has an idea?
Thanks in advance.

Hi all,
since my last post, I have further investigated using Wireshark to try and see what is the difference between the ethernet frames sent in both cases : when compiling with the Maple IDE and when compiling with the Arduino+STM32 addition.
I am really astonished of what I see.
With the Maple IDE, when I send a string like “Hello World\r\n”, I get 3 frames :
“Hello World”, then “\r”, then “\n”.
Which is already questionable.
But when I compile with the Arduino IDE, I get EXACTLY ONE FRAME FOR EACH CHARACTER !!
I have compared the Ethernet libraries of both environments, and basically they are doing the same thing, and they initialize the W5200 with the same parameters.
So what??

Timing?

Try putting in some delays, and slow the SPI to a slower clock rate, as its possible that the module is not being configured the same

I had a similar issue recently where on the Maple mini, on a different module, where a delay was needed between CS going low and the SPI transfers.
Also I needed to put a gap between each SPI byte, as the module was not able to cope with the ReadRegister command followed immediately by the SPI.transfers to read out the data.

It was almost working without the changes, but strangely, it had dropped 1 bit of incoming data, possibly the msb on a 32 bit register read.

Hi all,
I enventually found the deep reason of the problem of the messages scattered across several ethernet frames.
The mechanism is as follows.
The class EthernetClient has 23 print() methods it has inherited from the class print following the hereafter inheritance chain:
print – stream – client – ethernetclient.
There are 3 write() methods also inherited from print. They have been overloaded by three methods defined in the EthernetClient class. Unfortunately, the method from the class EthernetClient is defined :
size_t write(const uint8_t *buf, size_t size);
whereas the corresponding method from the class Print is defined:
size_t write(const void *buf, uint32 len);
As a result, the overloading does not operate.
Anyway, this is not the reason. It lays in the fact that every one of the 23 print() methods eventually call
size_t print(char);
from class Print, which in turn will call
size_t write(uint8_t);
from class EthernetClient,
the latter method being called once for each of the characters produced by the chosen variant out of the 23 print() methods.
This method builds one complete ethernet frame each time it is called, Thus any call to any print() method will be scattered into as many frames as the number of characters to be sent.
This leads to a very inefficient transmission, and in my case, to a system totally disturbed and unusable.
The right way to do it would be to open a buffer, put all the generated characters into it (including the final CR LF for the println methods), then submitting this string to the method
size_t write(const uint8_t *buf, size_t size);
from class EthernetClient.
However, I do not see any simple means to do it, except by overloading nearly every print() methods in the class EthernetClient.
What do you think about it?
Would you recommend to use the inherited print methods when performance is not of a problem, and use sprintf + write (buf, len) explicitly in the other case? Or do you think a new set of print methods should be written?

Is the void *buf argument in the Libmaple print function different from the official Arduino code e.g. for AVR or SAM boards, I guess it may be?

Did you try changing the void *, in print , to see if Libmaple still compiles, i.e are there places in Libmaple that pass void * to that function, which give an error or a warning if the print function is changed ?

Libmaple’s Print class is identical to that of Arduino, except that in the Arduino version, every function returns the number of characters printed, whereas libmaple’s functions are of type void.
So the same problem exists in any case: one character in one frame.

OK

I remember I fixed some libmaple serial functions which did not return the number of bytes written etc, but I must have not noticed all of them

If you change the function to return number of bytes, does it fix your problem ?

If so, can you create a PR with the necessary changes?

The problem has nothing to do with the functions returning or not the number of characters printed.
As I said:
It lays in the fact that every one of the 23 print() methods eventually call
size_t print(char);
from class Print, which in turn will call
size_t write(uint8_t); from class EthernetClient
This method send a frame each time it is called, for just one character.
Thus any call to any print() method will be scattered into as many frames as the number of characters to be sent.
This behaviour is fine for just printing characters, but inappropriate for sending Ethernet messages that can hold more than 1000 characters in a single frame.
What we need, is that the calls to print(char) do not activate the frame sending, but merely store the produced characters in a buffer, until a call to a special method sends the buffer at once.
However, I do not see how to do this without modifying the Print class, which belongs to the core. Do you think we are allowed to do this (for sake of compatibility with the current Arduino developments, I mean)?

@phono

Would the Ethernet library do the same thing if it was run on Arduino SAM or AVR boards, or is their code different, so it doesn’t have this issue?

I’m not averse to changing the print class to match with the Arduino SAM core does, (albeit as long as this doesn’t cause a massive ripple effect across the whole of libmaple_

Strange, I have an Arduino compatible board with ATmega128A and WIZ5500 module over SPI, it sends several bytes (I use up to 150) in one TCP packet…

Ok, I think there is an ambiguity in the names of the libraries.
When I speak of libmaple, I speak of that which comes with the Maple IDE. When I say Arduino code, I mean that from the path
D:\ArduinoForSTM32\arduino-1.6.9\hardware\Arduino_STM32-master\STM32F1\
The print() methods from the print class only differ from the fact that in the original IDE all print() methods return nothing.
As far as stevestrong’s remark, I have looked at the print class from the AVR core.
Did you print strings to the Ethernet?
There it is different. The print(*char) methods and all its variants actually include the following call
write((const uint8_t *)str, strlen(str));
The write(*buf, length) method is overloaded by one from the EthernetClient class which actually sends the string at once, then sends the Ethernet frame.
So this is a major difference, which is not difficult to change in Libmaple.
However, this only applies to strings. When printing numbers (either integer or float), they are converted to characters and sent one character at a time, that is, one ethernet frame for each character. This holds true for both the original Arduino library and Libmaple.
To summarize, It is easy to fix the problem in Libmaple for the strings, but the problem remains for printing numbers.
I think it is a very crude way to handle Ethernet communication, which is then no faster than a mere serial link.
Of course, one can use Sprintf(buffer, format, variable, …) then print the buffer using the supplied print method. But still, I think it is a pity to have print methods that do the things so inefficiently.

In my code I use both versions:
eth.print(*buf);

I am surprised that the call to
eth.print(*buf);