ODIn/spark-instrumented-optimizer
1
0
Fork 0
Code Issues Pull requests Projects Releases Wiki Activity
85 commits 6 branches 181 tags 397 MiB
Commit graph

85 commits

This branch
This branch
All branches
Author SHA1 Message Date
Mosharaf Chowdhury 50612ab3a9 Every thread is a daemon thread => Program shuts down ASAP once its done. 
Some log messages added to blank exception handlers.
 2010-11-09 16:44:42 -08:00
Mosharaf Chowdhury 6cb76d9708 Commented out some cluttering log messages. 2010-11-07 19:16:03 -08:00
Mosharaf Chowdhury cbce44304c Added exception handling to 'sendStopBroadcastNotifications' 2010-11-06 19:37:27 -07:00
Mosharaf Chowdhury c37c74919f Bug fix + minor changes 2010-11-06 16:54:55 -07:00
Mosharaf Chowdhury 878d157ce3 Graceful shutdown after a single transmission in the swarm is over. 
There might still be a problem with the Tracker shutdown. It must be done explicitly by SparkContext.
 2010-11-04 22:09:14 -07:00
Mosharaf Chowdhury 10fc66b1c4 Print reception messages for duplicate blocks as well... 2010-11-01 14:00:40 -07:00
Mosharaf Chowdhury 03d73dea9f Refactoring... 2010-10-28 15:09:58 -07:00
Mosharaf Chowdhury 681dba1373 Removed some deprecated speed related code. 2010-10-28 14:40:49 -07:00
Mosharaf Chowdhury bcc7d38ace Added SpeedTracker to keep track of reception time per block from each source a peer has ever talked to. 2010-10-28 14:37:59 -07:00
Mosharaf Chowdhury 44eb79f64a - Measuring approximate time to receive a block from the sending peer. 
- Added some Exception handling/suppression code.
 2010-10-27 17:41:37 -07:00
Mosharaf Chowdhury ae2c93a54b Fixed some configuration bugs 2010-10-27 16:18:26 -07:00
Mosharaf Chowdhury e4dc7847ba - Receivers now send back the latest hasBlocksBitVector after every reception 
- Will probably settle for the push model. Still not removing the pull related codes
 - Refactoring...
 2010-10-27 15:36:58 -07:00
Mosharaf Chowdhury 059b6ae475 Added data structures to turn transmission to a pull model from a push one. 2010-10-27 15:05:29 -07:00
Mosharaf Chowdhury f7364111bb TalkToGuide breaks once the peer has all the blocks. 2010-10-26 21:26:45 -07:00
Mosharaf Chowdhury 8587353b3e Added some tuning parameters. 2010-10-26 21:16:37 -07:00
Mosharaf Chowdhury d70f3549ff MaxTxPeers and MaxRxPeers set upper limit on concurrent sending and receiving threads for every peer. 
Relevant timeout code is also in. There is still an exception to take care of.
 2010-10-26 17:53:03 -07:00
Mosharaf Chowdhury 1877f1c38d No one is waiting on hasBlocksLock. Removed... 2010-10-26 16:18:13 -07:00
Mosharaf Chowdhury bc1c51817c Beautified code... 2010-10-26 16:08:22 -07:00
Mosharaf Chowdhury d673be0e06 Minor changes. Still need to run on Mesos+EC2 2010-10-26 15:07:03 -07:00
Mosharaf Chowdhury 3f96c9b7e8 Resolved some bugs. Apparently, objects deep inside other objects could be passed as references. Bad Scala! 2010-10-26 12:56:27 -07:00
Mosharaf Chowdhury e659efca88 Added a synchronized block around numCopiesSent manipulation. 2010-10-26 08:52:56 -07:00
Mosharaf Chowdhury dba92f7dbe - Fixed bugs with storing received blocks in appropriate indices. 
- receiveBroadcast now returns, but still no good way to stop threads from running.
 2010-10-25 17:34:25 -07:00
Mosharaf Chowdhury 96797cc997 Replaced nested variable.synchronized blocks with a single top level synchronized { } 2010-10-25 14:53:56 -07:00
Mosharaf Chowdhury 2099aadbbd Implemented rarest first policy. This isn't working as expected (i.e., not distributing load as expected), probably due to some issues with how we pick peers to talk to. 2010-10-25 14:48:11 -07:00
Mosharaf Chowdhury 7a7123b525 Running in local Mesos. Multiple things have to be fixed though. Go through the TODOs... 2010-10-22 23:43:11 -07:00
Mosharaf Chowdhury 27362c6523 Fixed ordering of function calls and thread creations. 
Changed Slaves to Workers :)
 2010-10-22 12:03:40 -07:00
Mosharaf Chowdhury 88264a33cf Peers actually exchange information between themselves. 
Don't exchange info about neighbors yet :(
 2010-10-21 18:12:30 -07:00
Mosharaf Chowdhury 8e6ed77724 Put synchronized blocks around BitSets. 
Added getLocalSourceInfo method that returns a SourceInfo based on local info.
 2010-10-21 17:58:42 -07:00
Mosharaf Chowdhury 22a47a9d16 PeerChatterController implemented. Now things should start working... 2010-10-21 17:06:25 -07:00
Mosharaf Chowdhury 86b2824101 Formatting... 2010-10-21 13:30:52 -07:00
Mosharaf Chowdhury 53bd64afe7 - Added TalkToPeer class which will allow peers to communicate between them. 
- Still need the controller class that will decide which peers to communicate with
 2010-10-20 17:27:02 -07:00
Mosharaf Chowdhury 05bca235a7 Changed ChainedStreamingBroadcast to BitTorrentBroadcast and BroadcastCS to BroadcastBT 2010-10-20 14:37:20 -07:00
Mosharaf Chowdhury e70755c2a2 - Implemented TalkToGuide class that talks to the Guide periodically 
- Changed listOfSources from Vector to ListBuffer
 2010-10-20 14:14:44 -07:00
Mosharaf Chowdhury b1c1e80026 - Added BitSet of pieces to SourceInfo. 
- Streamlined code after removing GuideInfo.
- Minor refactoring.
 2010-10-19 17:53:44 -07:00
Mosharaf Chowdhury b45dd8b8a4 Using SourceInfo instead of a separate GuideInfo case class. 2010-10-19 17:18:20 -07:00
Mosharaf Chowdhury 365012f586 - Replaced pqOfSources with a vector: listOfSources 
- Guide now returns back multiple Sources instead of just one
- Receiver part is not updated though. So it wont work
- Added BroadcastCS.MaxPeersInGuideResponse
- Changed Source selection to simple round-robin
 2010-10-19 17:02:55 -07:00
Mosharaf Chowdhury 905745707c Removed ReplicaID from SourceInfo. 2010-10-19 15:35:54 -07:00
Mosharaf Chowdhury 4ad6c5218e Removed some commented sort/compare funtions from SourceInfo 2010-10-19 15:31:27 -07:00
Mosharaf Chowdhury 5deb23df96 Starting major changes to make it more like BitTorrent. 2010-10-19 15:28:36 -07:00
Mosharaf Chowdhury 744d49bc40 Receivers now send their hasBlocksBitVector to the sender. 2010-10-18 14:55:35 -07:00
Mosharaf Chowdhury 0791e20bfc Added hasBlocksBitVector to keep track of which blocks a peer actually have. 2010-10-18 14:34:33 -07:00
Mosharaf Chowdhury b02ad04560 Updated log4j.properties to print milliseconds. 2010-10-18 14:06:48 -07:00
Mosharaf Chowdhury 1af0cdbac7 Removed all SplitStream-related things. 2010-10-18 11:40:21 -07:00
Mosharaf Chowdhury a4c0281902 sendObject now takes parameters instead of relying on class 
variables.
 2010-10-14 15:36:23 -07:00
Mosharaf Chowdhury a137ca75da Got rid pf dualMode. 2010-10-13 17:01:00 -07:00
Mosharaf Chowdhury 38194e5731 - Changed guidePort to GuideInfo that now contains the hostAddress 
as well as the port. This will allow anyone other than the master
to be a guide.
- The GuideInfo object now contains the constants related to
tracker response.
 2010-10-13 16:26:18 -07:00
Mosharaf Chowdhury 8690be8f5a Cleared up some formatting. 
Branching out from here to work on BT.
 2010-10-13 11:40:03 -07:00
Mosharaf Chowdhury 0d67bc1cee multi-tracker branch now compiles and runs; but it crashes right before the 
end. The same problem is seen also in the master branch (in the
ChainedStreaming implementation)
 2010-10-12 15:39:53 -07:00
Mosharaf Chowdhury 4fdd48295b Added mesos.jar. Still not working. Major changes required. 2010-10-12 13:10:31 -07:00
Mosharaf Chowdhury e73a5f3491 Now compiles with Scala 2.8.0, but doesn't run with nexus.jar 
Must update it to use mesos.jar
 2010-10-12 13:05:32 -07:00