This is the year where things started. I managed to get downloads working little more than a week before 2003 ended. I thought achieving this would let me take a break from improving MT but that is not quite how things went in early 2004...
2003 - December
2003 - November
- 19h30: Ok, now this should really be the last for this year :)
- Fixed a visual bug in the torrent page - the total uploaded/downloaded are shown as rates instead of plain amounts.
- Added the "Host" tag to HTTP requests, this fixes PHP trackers' 404 errors due to HTTP page caching.
- Added preliminary fragmented (multi-packet) tracker response support. (Untested since my TCP/IP stack is not setup to prevent fragmentation so I get all my requests in a single ~4KB TCP/IP packet... but single-packet responses are working so I can at least confirm that I most likely have not broken something that was known to work.)
- The first 2004 build will be alpha1.
- 03h28: Re-oops - some animes came out and some pretty annoying bugs popped up while trying to download them...
- Just noticed that the veryprealpha4 link actually pointed to veryprealphav3... oops :)
- While adding the directory selection code for the multi-files torrents, I accidentally killed the file names for single-file torrents!
- Discovered a typo (= where a == should have been) in my BEncode parser that cause it to crash if a torrent file contains zero-length keys. This should fix a few crash-on-.torrent-load issues.
- For some reason, MT is having a hard time getting peer lists from some trackers...
- 04h50: Upon further investigation, I finally found out that I wrote the tracker request code before re-discovering the end-of-line sequence... which one comes first? 0x0D or 0x0A? Which one is CR, which one is LF? I keep mixing them up but at least I have re-discovered that the proper sequence is 0x0D 0x0A.
- 05h10: Now that yesterday's build-pulling issues have been resolved, I can now post veryprealpha5v2, go to sleep and hope nothing else will go wrong today :)
- Fixed a few minor bugs.
- Enabled preliminary multi-file torrent support and some file access bugs revealed themselves in the process so that's that many more potential bugs fixed for single-file torrents.
- More more-and-less nasty bugs discovered while implementing some new stuff so more fixes followed.
- Latest build for this year, pretty much for sure this time:
- Actually yes and no since I pulled the file due one stupid bug, one discovered bug in old code and the issue connecting to one specific tracker I would like to solve before I go to sleep, see 2003-12-31...
- I ended up posting this stuff at 01h15... spent too much time rewriting stuff in my pages.
- 20h50: Oops, more fixes on the way...
- Fixed the request queue running dry, bug introduced while rewriting parts of the piece request scheduler.
- Removed periodic field sending.
- Fixed a bunch of problems that appeared after disabling periodic field sending.
- Uncovered a few major flaws to justify another build and the 'veryprealpha' label, the most important of which is malformed "Have" packets in which I apparently forgot to insert the message type leading to instantaneous disconnect every time Haves are sent.
- 03h05:
- Various (many) crash-bug and glitch fixes.
- Tracker URL escaping was missing a few low-ASCII characters. This should fix a number of cases where trackers do not respond for no apparent reason.
- Tracker retries are every 60 seconds until the first successful request - previous builds were re-connecting every second due to an incorrect initializer.
- Request timeout counters are reset upon data reception reduce the "Junked data" occurences due to request time-outs.
- Drag&Drop .torrent files to start downloads.
- Changed the throttler controls a little.
- Remember the last selected directory as the default for the next torrent.
- Added Completion Time, ETA, Completed Copies and Share Rating to the torrent summaries, I will try to find time to add these stats to the torrent pages as well before the next build after this one.
- You probably guessed it by now, enough new and improved stuff to justify another minor build...
- Got a Radeon 9600XT so I'll probably spend most of the day running DX9 demos with all the eye candy turned on to see what all the hype is all about. I'm not a gamer but the 400MHz 10bits main DACs is a visible improvement over my Rage128Pro and the R8500 I had. All I need now is a DVI-VGA adaptor and eventually an extra monitor.
- Since it will be another 2-3 days before I can put together the next set of (more significant) changes (assuming the least amount of disruptions), I have decided to post my current build which has many minor fixes and tweaks that should already increase MT's usability by a fair amount over reallyprealpha1...
- Merry Christmas!
- Drive safely!
- Maybe a new build tomorrow or the day after that with a couple of minor fixes all over the place.
- Things I forgot to mention yesterday about the latest MT build:
- No gzip tracker response support yet - this means some trackers may refuse to reply or return an error.
- MT apparently has some issues with some .torrent files.
- On some system, opening an existing file generates a 'File not found' error even though the file opening options are set to create the file if it does not exist. This sometimes happen even with existing files.
- New things for today:
- Doing some more code re-structuring to finish completely dissociating the MT core from the rest of the code - I was using globals for logging and timing facilities, now I'm moving them to passed down through class constructors. This is also to cut down on redundant #include-ing that can slow down compile times quite a bit by triggering unnecessary file rebuilds.
- Did a few piece request tweaks, outstanding pieces should not be stalling nearly as long as they were. Also reduced the piece request size to 16KB since many pieces were being junked due to request time-outs. Also added some sanity checks for tracker initialization.
- Now working on the new piece management classes, a new MT build using it whould be released in 2-3 days.
- Registered the domain name "MoonlightTorrent.com" so you can now access this site by going to
MoonlightTorrent.com
- Fixed a glitch in the input buffer that caused it to backlog messages by processing only one of them per OnReceive event, causing requests and other packets to be delayed until KeepAlive message flushed them out
- Rewrote the request scheduling code
- Basic download message scheduler
- Initiate connections only while downloading, only while there is spare download bandwidth
- Randomized connection timers to avoid bulk connects - spread out connects and initial status changes over a minute.
- In principle MT should be able to send piece requests now so unless I discover my code needs a major sanity check when I test downloads tomorrow, the first download-capable build should be out tomorrow.
- Rewrote minor GUI stuff all over the place.
- Implement some missing functions I thought I had already done.
- Fixed various minor crashbugs.
- And the moment you have all been waiting for:
One week over a month and 300h of worth of work later:
The first download-capable build! (238KB)
Far from perfect but it does do a decent job of downloading stuff!
Brave souls may download without reading the alpha warning/notes stuff below by clicking the link above.
- Very very alpha stuff, use with caution.
- Download and upload scheduling is nearly brain-dead.
- Bandwidth management is minimal - particularly download-wise.
- The downloader currently does not do pipelining, this will most likely mean more chokes and lower speeds.
- Several unwritten rules of the BT protocol remain to be discovered so stalls and other unpleasant things will happen regularly. (and stalls do appear to happen quite frequently... most likely because MT does not do request pipelining yet.)
- The current implementation is rather wasteful due to the previous item.
- I have tested this on my LAN, I get at most ~30KB/s from each client, this bottleneck is most likely due to not pipelining.
- The current implementation pre-allocates up to 20 piece buffers... this means up to 20 active pieces, which means 5MB/torrent with 256KB pieces. The download system that will be put in place at the same time as pipelining will be enabled will be much more RAM- efficient.
- The current piece splitting... is non-existent so, one piece per peer and the piece is stuck with that peer until it is either completed or its inactivity timer times out so the piece can be re-assigned.
- Lots of other stuff that can probably go wrong.
- Lots of hard-coded stuff that should not be.
- Lots of things I will try to smooth out a little before the year ends and release a veryalpha2 if I can - at the very least make the download scheduler smarter.
- Multi-file torrents are not supported yet.
- Basically, I'm posting this to show I have something that is >>>STARTING<<< to work both ways.
- If you're brave enough, click the link below later today. (05h10: I have decided not to post this first build immediately because it has a major memory usage issue so I will fix that and one or two other things first. The file should be up later today unless I find something worse than the stuff enumerated above.)
- 22h00: Ok, I'm done for today, you can try MT-veryprealpha1 if you wish. Given how much of a difference any minor improvement in the protocol handling, there will most likely be one or two more builds before 2003 ends.
I was not happy with my initial bandwidth control code so I started a minor rewrite to make it more uniform across bandwidth-consuming classes. This will make it easier to manage default bandwidth/connections-related parameters later. Some more work on the download chain was done yesterday but most of the day was spent writing my MoonlightTorrent Protocol - Preliminary Specifications (rewrite pending [2004-04-17]) page.
- 2003-12-17: "Streamy" sockets appear to be working properly
Uploads are back in working order, a few bugs and chatter stalls have been cleared up, bandwidth allocation and network IOs are done in finer increments and MT can now upload to both Shadow-587 and Azureus clients... after playing with Azureus, I understand why some people asked me to look into porting Azureus to C++... it is a really nice client but can be somewhat heavy on CPU/RAM usage. Once I will be done implementing all the functionallity I really want in MT, I will probably start borrowing some ideas from Azureus. - Now to ??? : Convert the socket handling code from packet-oriented to stream-oriented, somewhat of a pain in the a** but it should make the rest of upload, download and other stuff's implementation easier once it will be done. In any case, this probably means an Xmas or NY 0.1.0 release.
- Uploads are back in working order, a few bugs and chatter stalls have been cleared up, bandwidth allocation and network IOs are done in finer increments. Upload functionality has been tested with both Shadow-5.8.7 and Azureus 2.0.4.3-CVS (http://azureus.sourceforge.net).
- After playing with Azureus, I understand why some people asked me to look into porting Azureus to C++... it is a really nice client but can be somewhat heavy on CPU/RAM usage. Once I will be done implementing all the functionallity I really want in MT, I will probably start borrowing some ideas from it.
Still a little dizzy from the medecine and other stuff. I updated my eMule build to 0.30e code base today for those who are interested and I will get back to work on MT tomorrow, which means you can probably expect a new build in 2-3 days and cross your fingers for a download-capable one this weekend (again) :)
- 2003-12-12: Not feeling like moving today - I helped moving furniture downstairs at my mom's place yesterday. This and the cold combined mean I hurt all over today. I'm guessing today was the worst of it and I should be able to start getting back on track tomorrow.
- 2003-12-10: Caught a cold so don't expect new news and new builds until I have recovered. The stream handling is almost complete so the only major thing left to do about it is to break my current MT socket handler, plug the streaming layer in then make it work again. [I had forgotten how annoying/debilitating major colds can be... the next 3-5 days are going to suck.]
- Now to ??? : Convert the socket handling code from packet-oriented to stream-oriented, somewhat of a pain in the a** but it should make the rest of upload, download and other stuff's implementation easier once it will be done. In any case, this probably means an Xmas or NY 0.1.0 release.
- Few more tracker code fixes.
- Few more crash fixes.
- Upload/Download speed indicators.
- Fixed display context resource leak - causes MT to crash after a few hours. (Doing GetDC without doing ReleaseDC makes Windows act like it is dying after a while.)
- Fixed initial tracker queue rotation and update delay period when multiple torrents are associated to one tracker.
- Removed some logging messages, started to add selective data dumping in the debug dump.
- Configurable global speed limits, per-torrent limits should be done tomorrow.
- 2004-01-10: still not done yet and no plans to do it in the near future since there are many more important things to do.
- 17h00: Another build - since MT now generates real traffic, I hacked together some bandwidth counters. I also increased the hard-coded speed from 5KB/sec to 10KB/sec. I'll work on this some more and make it configurable monday. I also turned off hex dump in the release builds.
- 15h48: Oops, wrong build (old) in the zip, updated to MT 0.0.8 now.
- Many more bug fixes
- No downloads yet :(
- Finally figured out why I was losing track of some sockets.
- More crash-bug fixes.
- Some GUI glitch fixes.
- Also realized that something happened to the file hashing code... I apparently dropped the two lines responsible for loading the hash table from the .torrent file while shuffling code around.
- Extra checks.
- Partially rewritten handshaking code to handle outgoing connection initiation.
- More bugfixing due to my fixes breaking previous fixes because I had forgotten the exact way stuff I wrote two weeks ago worked and hadn't looked back into since was working :)
- Started to implement the resizable interface. I'm quite happy with the way the interface has gone so far and the only major "would be nice" I could think of is the resizability since I (and probably most of you out there, me included) hate scrolling just because the window is slightly too small and fixed-size.
- Fixed a number of bugs here and there, rewrote many things, maybe my new build will work on more PCs now!
- The new demo build (still no downloading yet - any day now, if not today) simulates a download to animate the progress bar so people can get an idea of how download progress will look.
- Updated screenshot to show what I mean.
- I wanted to do a sub-piece manager so I could have multiple sources working on a piece and keep RAM usage down... but the range management gave me headaches so I decided to go with one-source-one-piece until I get the downloads working. I'll see about implementing ranges after the core functionality has been fully tested and the [un]choke/[not]interested/have/field exchanges have been reasonably well tweaked.
- Just for fun, I checked CPU usage for the GUI whole neither downloading or uploading... on my P3-1G/512MB/WinXP, usage is 0-2% on the torrent list page and 5-8% in a torrent page. On my P120/48MB/Win98SE laptop, usage is <5% on the torrent list page and 20-80% on torrent pages, a nice change from the constant 100% and endless swapping I was getting with Python clients!
- Actually, I was testing on the P120 to see if CPU speed was responsible for the stability issues that came up on my friends' PCs and after hours of running MT on my P3-1G and P120, if there is a speed-related issue, then it seems it shows up only on PCs faster than mine and I won't be able to resolve them until I upgrade my PC, possibly about half-way through 2004.
- Ended up breaking a lot more so I spent most of yesterday fixing everything.
- Still working on the piece requesting code... I hoped to be done with it yesterday, but that was before the breakage.
- Availability graph will double-duty as progress bar...
- Some of the indicators are still not-implemented for the moment.
- Almost all the GUI bugs have been ironed out, along with several MT core issues.
- The current demo computes a nice little graph to see how evenly (or not) seeded the file is, see updated screen grabs below.
- As I make my way towards a download-capable client, I keep finding out I need something more to make it work... but I think most of these unexpected discoveries are done now and only the piece request handling remains to be coded to get a functionning BT client. I expect to be through with it this weekend.
- Memory leaks are gone - turned out all of them were 'ghosts' due to incorrect crtMemDump usage, meaning I wasted a couple of hours tracking non-existing leaks! Some of the stuff is not implemented yet... as mentionned previously, I'm only laying out the GUI and implementing elements useful for testing the core's functionality.
- I know what object is being leaked but cannot find how the leaking objects are created. Not too much of an issue since they are in the timer chain and should be in the "Expired" state and be garbage-collected (deleted) after callback.
- Virtual file handling is working - at least with single-file torrents.
- More network-related stuff implemented, only the piece requesting/sending left to implement.
- Piece (progress) tracking bitfields and request preparation.
- I felt like taking a break from low-level stuff and felt like looking into GUI stuff. The prototype GUI is now a property page.
- New memory leaks have appeared and I still have no idea where they come from.
- The P2P protocol should be functionnal soon.
- Listener socket(s) management is done.
- Tracker stuff is done.
- About 100h spent on this project over the last week to get from nearly nothing to almost functionnal!
- Compute torrent hash and session ID.
- Connect to trackers.
- Build peer handshake packets database for later analysis when I get to the CTC protocol.
- Parse .torrent files v2 : much cleaner key database implementation - accidentally broke the previous one.
- Basic socket handlers are in place, only need to add the protocol now.
- More flexible scheduling thread, in case I ever need it.
- Parse .torrent files to extract data,
- Skeleton implementation of pretty much everything I am expecting to need.
Hits since December 5, 2003:
Generated on Tue Aug 24 23:57:31 2004 for MoonlightTorrent(.com) by
1.3.8