Some BitTorrent

notes.org

@@ -281,7 +281,7 @@ than one hop away from the destination), but there is no routing table entry. As
 - A backhoe is unlikely to take out a major part of the system – at least if we store at k closest nodes
 
 * Mobile Ad-hoc Networks and Wireless Sensor Networks
-TODO : READ COOPERATION WITH OTHER PROTOCOLS IN MANET TEXT
+TODO : Finish the survey on sensor networks. I stopped when they started talking about actual schemes, as Bouvin didn't mention this in his presentation
 ** Routing in Mobile Ad-hoc Networks
 *** Introduction
 - Routing is the process of passing some data, a message, along in a network.
@@ -591,7 +591,7 @@ TODO : READ COOPERATION WITH OTHER PROTOCOLS IN MANET TEXT
 - Non-coordinator nodes are thus not used for routing purposes and they may therefore spend some of their time sleeping.
 - A coordinator selection scheme attempts to distribute the coordinator responsibility among the nodes.
 - Nodes have battery capacity and utility. Utility being it's reach in the network.
-- The coordinator selection algorithm is invoked periodically at every non-coordinator node
+- The coordinator selection algorithm is invoked periodically at every non-coordinator node. The result is a delay before the node becomes a coordinator.
 - So is a coordinator-withdrawal algorithm, at the coordinators.
 - the potential coordinator node needs information about its one and two-hop neighbours, and for each neighbour also whether that neighbour is a coordinator. This information is maintained pro-actively by using a standard HELLO message approach, as the one described, where each HELLO message contains information about neighbours and coordinators of the sending node.
 - As mentioned both the utility of the node and the remaining energy is taken into consideration when finding new coordinators. The way that it is implemented is by using a randomised back-off delay that the node uses before announcing itself as a new coordinator.
@@ -611,6 +611,126 @@ TODO : READ COOPERATION WITH OTHER PROTOCOLS IN MANET TEXT
 - We make coordinators be the only nodes who can forward RREQ. RREP simply follow the reverse path, as such no need to worry there.
 - There is no need to perform retransmissions, since the coordinator nodes are always awake.
 - A larger ratio between T_l and T_s is allowed, which results in lower energy consumption.
+** A Survey on Sensor Networks
+- The development of low-cost sensor networks has been enabled
+- Can be used for various application areas (health, military, home).
+- MANETs are intended to handle ad hoc communication from one arbitrary node to another
+- Wireless Sensor Networks (WSN) is about sensing, collecting, and shipping data in one direction—the sink
+*** Introduction
+- A sensor network is composed of a large number of sensor nodes, that are densely deployed either inside the phenomenon or very close to is (phenomenon ???)
+- The position need not be engineered or predetermined -> allows random deployment
+- Means the networks must possess self-organizing capabilities
+- Sensor nodes are fitted with an onboard processor, which allows for carrying out simple computations and thus transmitting only the required partially processed data, rather than all the raw data.
+- For military, it helps the network has rapid deployment, self-organization and high fault tolerance.
+- They require wireless ad hoc networking techniques. Many exists, but they aren't well suited to the unique features and application requirements of sensor networks.
+- The difference between sensor networks and ad hoc networks:
+  1) The number of sensor nodes in a sensor network can be several orders of magnitude higher than nodes in an ad hoc network
+  2) Sensor nodes are densely deployed
+  3) Sensor nodes are prone to failures
+  4) The topology of a sensor network changes very frequently
+  5) Sensor nodes mainly use a broadcast communication paradigm, whereas most ad hoc networks are based on point-to-point communications
+  6) Sensor nodes are limited in power, computational capacities and memory
+  7) Sensor nodes may not have a global identification ID, because of the large amount of overhead and large number of sensors
+*** Sensor Networks Communication Architecture
+- Sensor networks are usually scattered in a sensor field (basically one big cloud of sensors)
+- The scattered sensor nodes has the capabilities to collect data and route data back to the sink
+- Routing can be via a multihop infrastructureless architecture (I'd imagine they just broadcast until they find a node who can contact the sink ??)
+- Design of the sensor network is influenced by many factors
+  1) Fault tolerance
+  2) Scalability
+  3) Production costs
+  4) Operation environment
+  5) Sensor network topology
+  6) Hardware constraints
+  7) Transmission media
+  8) Power consumption
+**** Design Factors
+***** Fault Tolerance
+- Sensor nodes may fail or be blocked due to lack of power, or have physical damage or environmental interference
+- Failure of nodes shouldn't affect overall task of network
+- Fault tolerance describes the ability to sustain sensor network functionalities without interruption due to node failures
+- The Reliability or Fault Tolerance is modeled using the Poisson Distribution
+- The Poisson distribution expresses the probability of a given number of events occurring in a fixed interval of time or space if these events occur with a known constant rate and independently of the time since the last event.
+***** Scalability
+- The number of sensor nodes deployed in studying a phenomenon may be the order of hundreds or thousands
+- Protocols should utilise the high density in the sensor network
+***** Production Cost
+- The cost of a single node is very important, as a sensor network consists of many.
+***** Hardware constraints
+- A sensor node consists of a sensing unit, a processing unit, a transceiver unit and a power unit.
+- Perhaps also application specific components (bluetooth, GPS, mobilizer, stuff like that)
+***** Sensor Network Topology
+- Perhaps 20 nodes/m3
+- Topology maintenance and change in three phases
+  1) Predeployment and deployment phase (Can be deployed one by one, thrown to the winds or even by rocket, stuff like that)
+  2) Post-deployment phase (Topology changes can be due to reachability (because of jamming, noise and such), available energy, malfunctioning)
+  3) Redeployment of additional nodes phase (Additional nodes can be deployed)
+***** Environment
+- Can be inside, outside, in the ocean, chemically contaminated field, wherever
+***** Transmission Media
+- Links can be formed by radio, infrared or optical media (or really whatever)
+***** Power consumption
+- Only space for limited power source
+- Nodes dying can force changes to topology and such, which can be costly.
+**** Protocol Stack
+- Combines power and routing awareness, integrates data with networking protocols, communicates power effeciently through the wireless medium and promotes cooperative efforts of sensor nodes.
+- Consists of physical layer (simple but robust modulation, transmission and receiving techniques), data link layer, network layer (routing data supplied by transport layer), transport layer, application layer, power management plane, mobility management plane and task management plane.
+
+***** Physical Layer
+- Long distance wireless communication can be expensive
+- The physical layer is responsible for frequency selection, carrier frequency generation, signal detection, modulation and data encryption. Thus far, the 915 MHz ISM band has been widely suggested for sensor networks.
+**** Data Link Layer
+- The data link layer is responsible for the multi- plexing of data streams, data frame detection, medium access and error control. It ensures reli- able point-to-point and point-to-multipoint connections in a communication network
+- The medium access control (MAC) protocol in a wireless multi-hop self-organizing sensor network must achieve two goals. The first is the creation of the network infrastructure. The second objective is to fairly and efficiently share communication resources between sensor nodes.
+***** Power Saving Modes of Operation
+- Regardless of which type of medium access scheme is used for sensor networks, it certainly must support the operation of power  modes for the sensor node.
+- The most obvious means of power conservation is to turn the transceiver off when it is not required. Although this power saving method seemingly provides significant energy gains, an important point  must not be overlooked is that sensor nodes communicate using short data packets.
+- The shorter the packets, the more the dominance of startup energy
+- Turning the transceiver off during idling may not always be efficient due to energy spent in turning it back each time.
+- As a result, operation in a power-saving mode is energy-efficient only if the time spent in that mode is greater than a certain threshold.
+- The threshold time is found to depend on the transition times and the individual power consumption of the modes in question.
+***** Error Control
+- Another important function of the data link layer is the error control of transmission data. Two important modes of error control in communication networks are forward error correction (FEC) and automatic repeat request (ARQ)
+- Forward Error Correction
+  + Link reliability is an important parameter in the design of any wireless network, and more so in sensor net- works, due to the unpredictable and harsh nature of channels encountered in various application scenarios.
+  + Apparently encodes and decodes the messages, which costs processing power. Ratio to determine if this is worth it.
+  +  The central idea is the sender encodes the message in a redundant way by using an error-correcting code (ECC).
+**** Network Layer
+- Traditional ad hoc routing techniques do not usually fit the requirements of the sensor networks due to the reasons explained earlier.
+- The networking layer of sensor networks is usually designed according to the following principles:
+  1) Power efficiency is always an important consideration.
+  2) Sensor networks are mostly data-centric.
+  3) Data aggregation is useful only when it does not hinder the collaborative effort of the sensor nodes.
+  4) An ideal sensor network has attribute-based addressing and location awareness.
+- Energy-efficient routes can be found based on the available power (PA) in the nodes or the energy required (α) for transmission in the links along the routes.
+- An energy-efficient route is selected by one of the following approaches.
+***** Maximum PA Route
+- The route that has a maximum total PA is preferred
+- The total PA is calculated by summing the PAs of each node along the route
+- it is important not to consider routes derived by extending routes that can connect the sensor node to the sink as an alternative route
+  + Otherwise, we could end up simply going through the highest possible amount of nodes, as this would lead to the largest PA, but it would be hella inefficient.
+- Can be useful however. This somewhat resembles the routing protocol where the strongest should carry the biggest burden.
+***** Minimum energy (ME) route
+- The route that consumes minimum energy to transmit the data packets between the sink and the sensor node is the ME route.
+***** Minimum hop (MH) route
+- The route that makes the minimum hop to reach the sink is preferred.
+- Note that the ME scheme selects the same route as the MH when the same amount of energy (i.e., all α are the same) is used on every link.
+***** Maximum minimum PA node route
+- The route along which the minimum PA is larger than the minimum PAs of the other routes is preferred.
+- This scheme precludes the risk of using up a sensor node with low PA much earlier than the others because they are on a route with nodes that have very high PAs.
+- I reckon the PA here refers to the individual nodes PA.
+- prefer path with the largest of the smallest PA along the route.
+***** Data-centric approach
+- In datacentric routing, interest dissemination is performed to assign the sensing tasks to the sensor nodes. There are two approaches used for interest dissemination: sinks broadcast the interest, and sensor nodes broadcast an advertisement for the available data and wait for a request from the interested nodes.
+  1) Either sinks broadcast what they want
+  2) Nodes broadcast they have something. Metadata is cheap to broadcast.
+- Data-centric routing requires attribute-based naming. For attribute based naming, the users are more interested in querying an attribute of the phenomenon, rather than querying an individual node.
+  + The areas where the temp is higher than 70 degrees, rather than the temp at this specific node
+- Attribute-based naming is used to carry out queries by using the attributes of the phenomenon. Attribute-based naming also makes broadcasting, attribute-based multicasting, geocasting, and anycasting important for sensor networks.
+- Data aggregation is a technique used to solve the implosion and overlap problems in data-centric routing
+- In this technique, a sensor network is usually perceived as a reverse multicast tree where the sink asks the sensor nodes to report the ambient condition of the phenomena. Data coming from multiple sensor nodes are aggregated as if they are about the same attribute of the phenomenon when they reach the same routing node on the way back to the sink.
+- With this respect, data aggregation is known as data fusion
+- In sensor networks, data can be aggregated, i.e., collected into bigger packets along the way the sink
 * Accessing and Developing WoT
 ** Chapter 6
 *** REST STUFF
@@ -1008,3 +1128,118 @@ access the resources. It’s a web server offering an OAuth API to authenticate
 - The delegated authentication mechanism of OAuth relies on other OAuth pro- viders to authenticate users and create API tokens. As an example, a user of a Thing can be identified using Facebook via OAuth.
 - You can implement access control for Things to reflect your social contacts by creating an authentication proxy using OAuth for clients’ authentication and contacts from social networks.
 
+* BitTorrent
+** Incentives Build Robutness in BitTorrent
+- BitTorrent file distribution uses tit-for-tat as a method of seeking pareto efficiency
+  + Pareto =  ingen kan opnå en bedre stilling uden at en anden samtidig opnår en ringere stilling
+*** What BitTorrent Does
+- When a file is made available using HTTP, all cost is placed on the host machine.
+- With BitTorrent, when multiple people are downloading the same file, they can upload pieces to each other.
+- This makes a hosting a file with a potentially unlimited amount of downloaders, affordable.
+- It has been attempted to find practical ways of doing this before, however, issues have been encountered in regards to what peers have what files, where these files should be sent and the systems tend to have issues with high churn rate, as peers usually don't connect for more than a few hours.
+- Also has problems with fairness, as the total download of all peers must be the total upload of all peers.
+- In practice it’s very difficult to keep peer download rates from sometimes dropping to zero by chance, much less make upload and download rates be correlated.
+- BitTorrent solves these problems.
+**** BitTorrent Interface
+- BitTorrent is easy to download and launch. This ease of use has contributed greatly to its adoption, as it doesn't take a computer scientist to understand it.
+**** Deployment
+- The publisher of a file decides if he/she wants to use BitTorrent for distribution
+- Downloaders will then use BitTorrent, as it's the only way of getting the file
+- There is a risk of downloaders ceasing uploading as soon as their download completes, however it's considered polite to leave it on. This is also a requirement for some trackers.
+- Usually the number of incomplete downloaders (people who do not have the whole file), increases very rapidly once the file is made available. This peaks and then falls off at a roughly exponential rate, as people finish the download.
+*** Technical Framework
+**** Publishing Content
+- To start a BitTorrent deployment, a static file with the extension .torrent is put on an ordinary web server. The .torrent contains information about the file, its length, name, and hashing information, and the url of a tracker.
+- Trackers help downloaders find each other
+- A tracker is simply a server which speaks HTTP, where you can send what you're downloading, what port you listen on and such
+- There has to be a seeder to begin with, as someone must have the full file.
+**** Peer Distribution
+- All logistical problems of downloading are handled between peers. Some information about download and upload rates are sent to the tracker, merely for statistics.
+- Tracker only HAS to help peers find each other.
+- Although trackers are the only way for peers to find each other, and the only point of coordination at all, the standard tracker algorithm is to return a random list of peers.
+- Random graphs have very good robustness properties. Many peer selection algorithms result in a power law graph, which can get segmented after only a small amount of churn. Note that all connections between peers can transfer in both directions.
+- random graph is the general term to refer to probability distributions over graphs. Random graphs may be described simply by a probability distribution, or by a random process which generates them.
+- A power law graph has many nodes with few links and a few nodes with many links.
+- In order to keep track of which peers have what, BitTorrent cuts files into pieces of fixed size, typically a quarter megabyte.
+- Hashes are used to verify integrity of files. The hashes are included in the .torrent file
+- Peers don't report they have a piece, until they've checked the hash
+**** Pipelining
+- BitTorrent further breaks the chunks of file into smaller pieces, such that it can request each piece and always keep a static number of these requested. This makes the download more smooth.
+**** Piece selection
+- Selecting pieces to download in a good order is very important for good performance. A poor piece selection algorithm can result in having all the pieces which are currently on offer or, on the flip side, not having any pieces to upload to peers you wish to.
+***** Strict Priority
+- Always finish a particular piece, before ordering subpieces from another
+- Essentially what we do in SilverStream. This allows for streaming.
+***** Rarest First
+- When selecting which piece to start downloading next, peers generally download pieces which the fewest of their own peers have first, a technique we refer to as ’rarest first’.
+- It also makes sure that pieces which are more common are left for later, so the likelihood that a peer which currently is offering upload will later not have anything of interest is reduced.
+- It also makes it easier for the seeder, as leachers will see the others already have the old pieces and as such they will download the new pieces from the seed. 
+- Additionally, if the seeder stops seeding, it's important that the entire file is circulating still. This is helped by the leachers downloading the "rare" pieces first, such that together, they have the entire file.
+***** Random first piece
+- An exception to rarest first is when downloading starts. At that time, the peer has nothing to  so it’s important to get a complete piece as quickly as possible.
+- Rare pieces tend to only be on one peer, so these would be downloaded slower, than pieces on multiple peers.
+- As such, random pieces are selected until a complete piece is made. Then rarest first is used.
+***** Endgame mode
+- Used in the end when a few sub-pieces are missing
+- Just starts broadcasting what it's missing, in an attempt to finish
+- In practice doesn't waste much bandwidth, as endgame is short.
+*** Choking Algorithms
+- BitTorrent does no central resource allocation. Each peer is responsible for attempting to maximize  own download rate.
+- Peers do this by downloading from whoever they can and then decide which peers to upload to.
+- If peers cooperate, they upload to each other, if they don't want to, they choke other peers.
+- Choking is thus a temporary refusal to upload, while they can still download. (I assume they mean the peer who chokes others, can still download from others)
+- The choking algorithm isn’t technically part of the BitTorrent wire protocol, but is necessary for good performance. A good choking algorithm should utilize all available resources, provide reasonably consistent download rates for everyone, and be somewhat resistant to peers only downloading and not uploading.
+**** Pareto Efficiency
+- Well known economic theories show that systems which are pareto efficient, meaning that no two counterparties can make an exchange and both be happier, tend to have all of the above properties.
+- In computer science, this is a local optimization algorithm in which pairs of counterparties see if they can improve their lot together. These tend to lead to global optima.
+- if two peers are both getting poor reciprocation for some of the upload they are providing, they can often start uploading to each other instead and both get a better download rate than they had before.
+- Peers reciprocate uploading to peers which upload to them, with the goal of at any time of having several connections which are actively transferring in both directions. Unutilized connections are also uploaded to on a trial basis to see if better transfer rates could be found using them.
+**** BitTorrent's Choking Algorithm
+- Each BitTorrent peer always unchokes a fixed number of other peers, default is four.
+- This approach allows TCPs built-in congestion control to reliably saturate upload capacity. TCP is used by bittorrent as transport protocol.
+- Decisions as to who to unchoke, is based on current download rate. (Calculating current download rate is apparently difficult, so a rolling 20-second average is used)
+- To avoid situations in which resources are wasted by rapidly choking and unchoking peers, BitTorrent peers recalculate who they want to choke once every ten seconds, and then leave the situation as is until the next ten second period is up.
+**** Optimistic Unchoking
+- Simply uploading to the peers which provide the best download rate would suffer from having no method of discovering if currently unused connections are better than the ones being used.
+- To fix this, at all times a BitTorrent peer has a single ‘optimistic unchoke’, which is unchoked regardless of the current download rate from it. Which peer is the optimistic unchoke is rotated every third rechoke period
+**** Anti-snubbing
+- Occasionally, a BitTorrent peer will be choked by all which it was downloading from.
+- This will suck, until it gets optimistically unchoked
+- When over a minute goes without downloading from a particular peer, BitTorrent assumes it is "snubbed" by that peer and doesn't upload to it, unless it gets optimistically unchoked.
+**** Upload Only
+- Once download is finished, BitTorrent switches to preferring peers which it has better upload rates to and also preferring peers which no one else happens to be uploading to.
+
+
+** Attacking a Sawm with a Band of Liars: evaluating the impact of attacks on BitTorrent
+*** Introduction
+- Peer-to-Peer (P2P) file sharing has become one of the most relevant network applications, allowing the fast dis- semination of content in the Internet. In this category, BitTorrent is one of the most popular protocols.
+- BitTorrent is now being used as the core technology of content delivery schemes with proper rights management that are being put in operation (e.g., Azureus Vuze)
+- Major companies like Warner Brothers, 20 th Century Fox and BBC are distributing content through it.
+- as the popularity of BitTorrent grows, so does the risk and impact of malicious attacks exploiting its potential vulnerabilities.
+- This paper identifies attacks on BitTorrent and evaluates their impact on downloading efficiency (and eventual success) of peers taking part on a session (called swarm).
+*** Related work
+- Several issues with selfish peers, who seek to contribute nothing or as little as it can
+- in our work downloading is not relevant to a peer, whose sole intention is to hinder a swarm (using the minimum amount of resources needed). The idea is to make use of false piece announcements and/or large number of sybils to slow down or, ideally, to prevent content from being distributed.
+*** The BitTorrent Architecture
+- The content (a set of files) to be distributed is described through a meta-data file whose extension is typically .torrent.
+- A tracker is a central element that coordinates a swarm and helps peers to find other peers in the same swarm.
+- The content published is organized in pieces, and these are subdivided into blocks.
+- Peers establish a connection with each other and exchange bitfields containing information about piece availability.
+- Three peers are marked as unchoked, and a fourth peer is chosen randomly for uploading between the connected ones. (optimistic unchoking)
+**** Peer Connection Policy
+- Peers obtain from a tracker the IP addresses from other peers in the same swarm. Peers that take part in the swarm periodically connect to the tracker and this way notify their presence. After a peer connects to the tracker, it informs the number of peers ( numwant ) that it wishes to receive, 50 by default.
+*** Subversion and Attack Strategies
+**** Sybil
+- The Sybil attack in P2P networks consists in a single peer presenting itself with multiple, virtual identities, usually with the aim of exploiting reputation-based systems.
+- Using this attack, a malicious peer (henceforth called a liar) may get to represent a substantial fraction of the P2P network, and thus compromise it.
+  + Apparently a Certification Authority is the best form of protection. I'd imagine this is because the peer would need a certificate for all of it's identities and this would be difficult to get?
+- In BitTorrent, identities are randomly generated. An attacker, therefore, may exploit this vulnerability and obtain multiple identities. 
+- Peers that refuse to cooperate can be banned, however it can be difficult to distinguish between people who won't cooperate on purpose from people who have connection issues.
+**** Lying Piece Possession
+- As discussed in Section 3, peers employ messages Bit-field and Have to inform peers about piece possession. Following the LRF policy, (correct) peers strive to increase uniformity in the amount of copies of each piece. A Piece Lying attack aims at destroying this balance.
+- a malicious peer does not adhere to the protocol and announces a piece it does not have (thus it is a liar).
+- Thus, it artificially increases the level of replication of a potentially rare piece, causing other peers to download more common pieces first. This could lead to the piece simply disappearing from the network, as no one keeps it circulating. This fucks up the swarm.
+- As a malicious peer does not wish to deliver the announced piece, it keeps other peers permanently choked.
+- The impact of attacks is generally more effective when performed by many peers acting in collusion. In the specific case of making a piece ever rarer, we expect more peers to make the attack more harmful. The more peers lie about a given piece, the more frequent it will appear to become, and thus in practice the rarer in fact it will be.
+**** Eclipsing Correct Peers
+