New ftp server design.
- Hi all,
I have been throwing a new anonymous ftp server design around in my head for the
last couple of days. This design should easily be able to scale to many
thousands of users. It will rely heavily on the mincore system call that is in
2.4 kernels. It will do the following:
1) Handle all data and control contections with one thread.
2) Pre-cache the listing info of the entire ftp tree.
3) Use memory mapping to transfer files and cache large amounts of data
4) Use clone to create a thread for each device found in the ftp tree.
Ie, if files are spread over 2 drives, create 2 threads. Since drives
are access in serial, there is no point in more than one io thread per device
5) When a disk read is forced (found out via mincore()), notify
the thread that is connected to the device the file will be read from, and
then the thread will do the io (access the data). The threads will also
use mlock() to make sure the data stays memory.
This architechure would require:
1) Lots of memory. Start to swap, and everything goes to hell from
2) A static document tree. This restriction could likely be removed at
a later date.
3) Does not scale really well with more than 1 processor. Again, could
be fixed at a later date.
However the benifits are:
1) Extremely fast performace, even when document tree > ram size.
2) No blocking of the main thread, due to disk reads being forced in
3) list caching means no big hit when listing files.
4) Fewer context switches than most other servers.
5) When a server is blasted due to only a few files, this server
should be able to handle it will due to data caching (no kernel calls
to read it from disk again + cache is locked into memory)
Are there any significant holes or problems with my idea that I haven't
BTW, I may never get around to fully implementing this, but i am about to have
a go now, downloading 2.4.0test4