The SDL forums have moved to discourse.libsdl.org.
This is just a read-only archive of the previous forums, to keep old links working.

We have a hardcoded '5' in the listen() call in SDLNet_TCP_Open(), so if
you have more than 5 people that have connected for which you haven't
yet called SDLNet_TCP_Accept(), they'll get a "connection refused" error
(or on some OSes, they'll just retry up to a certain time limit). I
don't believe there's an actual maximum number of accepted connections
beyond what the OS allows, so just be sure to check for new connections
once per frame and you shouldn't have an issue.

UDP is connectioness, so this is only an issue with TCP.


I believe this is OS-dependent for UDP, and for TCP they never get
dropped, since you have to have a reliable stream here.

However, and I'm just guessing here, but there will probably come a
point where the OS tells the remote host that its internal buffers are
full and the remote host has to block and continue to resend until it
gets through...this happens transparently to the apps in blocking mode
and probably just gives a short write on the remote host for
non-blocking mode.

Just read data every frame and you almost certainly won't run into this
issue.

UDP packets aren't guaranteed to arrive, so I assume if the OS runs out
of internal space, it either drops the oldest queued packet or drops the
newly-arrived one until you start reading again. Again, just read all
new data every iteration and you'll be fine.

--ryan.

That's interesting. My application suffers from the fact many
connections may arrive at once on a host. I guess it could be easy to
hack SDL_net's source to increase this limit to reach at least the
OS's limit.


On TCP, packets do get dropped, but it is the OS's job to retransmit
them. This is called congestion. On a very busy host, when lots of
packets arrive in disorder, the host has to hold all that unordered
data until the missing pieces arrive so the server can reorder the
stream. The amount of data the server can hold unordered is the TCP
window. But because the window is set for "the perfect world" and in
our world windows do get full and use a lot of resources. Many
protocols have implemented a congestion-window on the sender side,
that is to send data at the maximum rate until packets start being
dropped (still talking about TCP here) then the congestion-window is
lowered. This just means the client understands the server window
adverstised may not equal the real-time window on the server.


Exactly, but my application needs to be aware of all this, I will
certainly have to use UDP to catch those problems and deal with them
my way.


My problem is the server of my application will get lots to process
every frame. I may expect some situations (maybe not so rare) where
every frame will take several seconds, maybe even more. During this
time, lots of trouble can happen as you may understand. Design-wise,
there is not much we can do about this except balance all that load
onto a different server, but this is just "distributing" the problem
and not solving it.


I guess this is done OS-dependantly. To my application, it doesn't
matter much, the fact is: 1) packets will get dropped, no matter
which, 2) those dropped packets will be resent 3) packets will be
resent until succesfully received.

Basically, my application will be compiled for different OS and I will
not be able to avoid the OS-dependent characteristics. The best I can
do is to make the application aware of the limits of its system.

Thanks,
Simon

Why not have seperate threads dealing with connections and
read/writes? They could put data into your own buffers, and you
wouldn't have to worry about OS limits.

Hello,
SDL is geat for all IO management. But when I had to design a complex
client-server network system, I did just rewrote my own one. Today all
OSs understand the "unix-netowrk-C-functions", which are actualy used in
SDL_net, except max os classic...
Well, as far as I know:
-win32
-liunx
-maxosx
do.
And it's enough for me. I love OpenBSD, and I guess it would work with it too,
but haven't tried. So I would simply advise you to use the direct unix C calls.
There is just with win32, you have to call some init at the beginning, but
that's far from complex.
Well, I did rewrote my network abstraction because I improve some code
"cleaness" because I could do it into C++ too. At the end I found it did worth.
Maybe if you write C it doesn't worth. Just my toughts.

Nuage


Simon wrote:

Simon <simon.xhz at gmail.com> wrote:


i solved this once and for all for me with a little socket class. i
spawn a thread for each connection, each with its own receive buffer.
the network protocoll consists of an 4 byte header and an up to 4GB
body. the socket is opened in blocking mode. first i read the 4 byte
header, then realloc the buffer if needed and finally read the body.
when the whole packet has been read, i call back the application
through a virtual method. (can also be implemeted via a normal c
callback function, does not matter.)

i saved myself from all the packet dropping/disordering by using TCP
though. this design scales very well with minimum latency.

best regards ...
clemens

Hello !




Would it be possible to post this class here ?



CU

"Torsten Giebl" <wizard at syntheticsw.com> wrote:


my very first C version is available under GPL already at www.1541.org
it's called SNL (Simple Network Layer). the C++ redesign, which is
somehow saver to use and has simple UDP support, is part of a library i
did for my current employer. i will try to set the whole library free
anyway, but i have to check with him first.

i will rip the socket class from the complete lib though, as it is
based on SNL anyway. give me a few minutes to package it ...

clemens

Just to play devil's advocate (and before this gets too far from
discussion of SDL_net): I assume that four-byte header is an unsigned
int that reports the amount of data that will be forthcoming, so you can
realloc if needed.

What happens when some jerk sends you a packet that starts with
0x4000000, and then feeds you a gigabyte of data? If he does this with
multiple connections at once, does the server just sort of fall over
when malloc() eventually fails, or does it bring the whole box to its
knees swapping before that happens?

You might want to put an upper limit on that currently-4GB maximum, and
assume it's malicious if it's more than, say, a kilobyte, and drop the
connection.

(If a kilobyte is too small, fine, 100 kilobytes, whatever.)

Network programming has so many nasty what-if situations. :)

--ryan.

"Torsten Giebl" <wizard at syntheticsw.com> wrote:


i have uploaded the socket class to my site:

http://www.1541.org/public/socket-0.0.1.tar.gz

the archive contains the socket.h and socket.cpp files and basic
demonstration server and client code (also for UDP). the socket.h file
is doxygen documented, so you can generate the API documentation if you
have doxygen installed. after untaring, just cd into the socket dir and
build with make.

have fun ...
clemens

"Ryan C. Gordon" <icculus at icculus.org> wrote:


right, for the time being, i'm even unsure if its int or unsigned. 2GB
is already more i could handle anyway.


i thought about this (of course ), but as this beast was used
internally only so far, and clients had to authenticate with
predistributed keys anyway, there was no need to handle this.
authentication and encryption is thus missing in the archive i uploaded
today. if there is demand for it and i will get permission to release
the rest of the lib this will be addressed.


for now this is left as an exercise to the fellow reader. ;-)


indeed. thank you very much for pointing this out. i should have put a
warning in the readme that the socket class "as is" could be a target
for DOS attacks.

anyway, currently i'm not even 100% sure the lib is without a race. it
has proven itself stable in all tests and never failed so far, though.

best regards ...
clemens

You are avoiding the problem with this solution. I would like to know
the problem and deal with it.

Besides, using thread in high demand servers is not recommended.

Thanks,
Simon

Can you tell me how you can handle more than 2000 connections to you
server, without the need for maxing out the amount of RAM in the PC?

True, that's why I was asking about the limits in the first place... it
seems nobody ever tried to find vulnerabilities in SDL-based code, or
are there anybody who tried?

Anyway, I will be working on a first draft soon and I will possibly try
to crash it all ways possible and find the limits the hard way. I
wonder how the base library is strong against buffer overflows, etc...

All this to avoid DoS.

Thanks,
Simon

Thanks Nuage,
That's very interesting... do those libs support more than SDL_net?
I guess so. Would I be able to do ICMP communications too?

Thanks,
Simon

On Sunday, January 08, 2006 1:38 AM GMT,
Simon <simon.xhz at gmail.com> wrote:


forget about maxing memory, how the heck is he able to handle up to 4GB body
the MTU is 1500 bytes on LAN (8KB if you use jumbo frames on GbE), not to
count all the TCP overhead, and still maintain "minimum latency"


Regards,
IraqiGeek
www.iraqigeek.com

Simon <simon.xhz at gmail.com> wrote:


i don't think there are any, at least not in the linux/win32 codepath.
SDL_net is a rather thin layer around bind/accept/connect/send/receive
so it does no buffering or threading on its own which is most error
prone in network code.


see above.

best regards ...
clemens

"IraqiGeek" <geek at IraqiGeek.com> wrote:


i don't see why 2000 should be the limit in any way. yes, if you expect
to have really many simulatanious connections to the server all the
time (tens or hundreds of thousends), another solution might be better
because each connection means an additional thread with its seperate
process context, stack aso.


how much data you send is up to you you, it's just that the library
does not force any unnecessary low limits on you. minimum latency
means your application gets the data as soon as it is available
(completely). of course there are other ways to implement this (select)
but i prefered to use the blocking thread method. having only thread
is nice but then, if you have high traffic on your server and packets
get bigger (fragmented) and thus read() does not return the whole packet
(of your higher level protokoll) at once (what you never can rely on
anyway), things start to get complicated.

i wanted two things from my network lib: packets should always be
complete when deliverd and receive buffers should not be memcopied.

best regards ...
clemens

I doubt it, but it's really a very thin wrapper over BSD sockets (unless
you count the OpenTransport code for MacOS 9)...frankly, if this is a
huge concern for you, I'd use BSD sockets directly.

--ryan.

Security is one of my main concerns in this project, but it's not
prioritary for the development. I'll probably keep working with SDL_net
and may switch over to those BSD sockets. They seem to do all I need
more directly.

Thanks,
Simon

IraqiGeek wrote:

I agree, this large buffer can never work on a high end server.
I've checked a couple of protocols and the best way I found to do my
project, is to use a combination of ICMP, TCP and UDP. All of them will
have small buffers since I prefer to drop information rather than being
DoS or maybe have to ignore more connections.

Thanks,
Simon

Simon <simon.xhz at gmail.com> wrote:



without wanting to sound rude, i don't think you have understod, how
this works.

initial buffer size is 4kb. if you do not send larger packets than 4kb,
the buffer size will never increase to more. even if somebody sends a
broken header, there is no problem yet, because mallocing a large
buffer without using it, will not claim more memory.

putting an upper limit of buffer size, depending on your high level
protocol is trivial.


sounds really complicated and i can not imaging how ICMP could be
useful here, but who knows.

clemens