Skip to content
Snippets Groups Projects
Commit 99a19ea0 authored by Nick Mathewson's avatar Nick Mathewson :game_die:
Browse files

Revise repository-level documentation

parent 93102734
No related branches found
No related tags found
No related merge requests found
Hide whitespace changes
Inline Side-by-side
CONTRIBUTING.md
+ 75
− 8
View file @ 99a19ea0
# Contributing
# Contributing to Arti
## Dogfooding
We welcome new contributors! You can get in contact with us on
[our gitlab instance](https://gitlab.torproject.org/), or on the
[`\#tor-dev IRC` channel on OFTC](https://www.torproject.org/contact/).
Make sure to familiarize yourself with our
[Code of Conduct](https://gitweb.torproject.org/community/policies.git/plain/code_of_conduct.txt).
A good first step to start hacking on arti might be to hook it up with your Tor
Browser. Please note that arti is still a work in progress and hence you should
consider that it **provides no security** at the moment.
The new-account process on our gitlab instance is moderated, to reduce
spam and abuse. (*Insert instructions for anonymous usage here*)
## Licensing notice
Unless you explicitly state otherwise, any contribution intentionally
submitted for inclusion in the work by you, as defined in the Apache-2.0
license, shall be dual licensed as above, without any additional terms or
conditions.
## Using Arti with Torbrowser
A good first step to start hacking on arti might be to hook it up with your
Tor Browser. Please note that arti is still a work in progress and hence you
should assume that it **provides no security** at the moment.
To do so, we will launch arti independently from Tor Browser. Build arti with
"cargo build --release" and after that launch it with some basic configuration parameters:
`cargo build --release`. After that launch it with some basic
configuration parameters:
$ ./target/release/arti -c "socks_port = 9150" -c "trace = false"
$ ./target/release/arti -c "socks_port = 9150" -c "trac e =false"
This will ensure that arti sets its SOCKS port on 9150. Now we need to launch
Tor Browser and instruct it to use that SOCKS port:
$ TOR_SKIP_LAUNCH=1 TOR_SOCKS_PORT=9150 ./start-tor-browser.desktop
The resulting Tor Browser should be using arti.
The resulting Tor Browser should be using arti. Note that onion services
won't work (Arti doesn't have them yet), and neither will any feature
depending on Tor's control-port protocol.
Enjoy hacking on arti!
## Where are some good places to start hacking?
You might want to begin by looking around the
[codebase](https://gitlab.torproject.org/tpo/core/arti/), or getting to
know our [architecture](./doc/Architecture.md).
More tests would always be great.
Parsing more Tor document types would be neat.
More documentation examples would be great.
Improvements or bugfixes to the existing code would be great.
Improving the look and feel of the documentation would also rock.
I've made a bunch of notes throughout the document in comments with strings
like "XXX" or "TODO".
There is a list of features that I wish other crates had in a file called
`WANT_FROM_OTHER_CRATES`.
Finally, check out
[the bugtracker](https://gitlab.torproject.org/tpo/core/arti/-/issues).
There are some tickets there labeled as
["First Contribution"](https://gitlab.torproject.org/tpo/core/arti/-/issues?scope=all&utf8=%E2%9C%93&state=opened&label_name[]=First%20Contribution):
that label means that we think they might be a good place to start out.
## Caveat haxxor: what to watch out for
Please don't assume that what you see here is good Rust: we've tried to
follow best practices, but we've been learning Rust here as we go along.
There are probably aspects of the language or its ecosystem that we're
getting wrong.
Almost nothing about this code should be taken as "final" -- I expect
that we'll need to refactor and move around a whole bunch of code, add a
bunch of APIs, split crates, merge crates, and so on.
There are some places where I am deviating from the existing Tor
protocol under the assumption that certain proposals will be
accepted. See [Compatibility.md](./doc/Compatibility.md) for more
information.
This code does not attempt to be indistinguishable from the current Tor
implementation.
README.md
+ 97
− 260
View file @ 99a19ea0
# arti: A Rust Tor Implementation
# Arti: reimplementing Tor in Rust
(I'm choosing this name for temporary purposes, but with a nod
to the fact that temporary names sometimes stick. Its name is
a reference to Tor's semi-acronymic origins as "the onion
router", and to the Latin dative singular for "art". It is
also short for "artifact". It has nothing to do with the
"rt" issue tracker or the "rt" media outlet.)
Arti is a project to produce an embeddable, production-quality implementation
of the [Tor](https://www.torproject.org/) anonymity protocols in the
[Rust](https://www.rust-lang.org/) programming language.
[See here for API-level developer documentation.](https://tpo.pages.torproject.net/core/doc/rust/tor_client/index.html)
Arti is **not ready for production use**; [see below](#status) for more information.
## Is this the future of Tor??!?11?!?
## Links:
It's not the future yet! ;)
* [Official source repository](https://gitlab.torproject.org/tpo/core/arti)
I'm hoping that with time I can turn this code into a minimal Tor
client, and then we'll see what the situation is, and whether we should
keep on working on it.
* [API-level developer documentation](https://tpo.pages.torproject.net/core/doc/rust/tor_client/index.html)
There are absolutely no guarantees about stability. Future versions of
Tor might break this; no users should expect source compatibility.
* [Guidelines for contributors](./CONTRIBUTING.md)
## What the demo can do if you run it.
* [Architectural overview](./doc/Architecture.md)
It can act as a SOCKS proxy that uses the Tor network. (Not a very
secure or anonymous one!) It knows how to download directory
information and how to load it from cache, but it doesn't try to
download more than one directory per run.
To try it out, run the demo program in `arti`. It will open a
SOCKS proxy on port 9150.
% cargo run --release
Again, do not use this program if you need anonymity, privacy, security,
or stability.
## What's here and what isn't.
So far the code has untested or under-tested implementations of:
* the ntor protocol
* the relay crypto algorithm
* parsing and encoding all the cell types (except for hs-related
ones)
* parsing and validating ed25519 certificates
* parsing and validating router descriptors
* parsing and validating microdescriptors
* parsing and validating microdesc consensuses
* link authentication (client->server type)
* building circuits from chosen hops
* a slightly wonky circuit abstraction
* a slightly wonky stream abstraction
* Downloading and caching directory documents
Before I share it more broadly, I think it needs more work on:
* refactoring everywhere
* cleanup everywhere
* lots of tests
* a better api for cell types
* better APIs for anything that needs to get exposed
There is no support yet for:
* Actually connecting to the network in a reasonable way
* choosing paths through the network in a reasonable way
* doing anything with the network in a reasonable way
* creating network documents
* v2 onion service anything
* v3 onion service anything
* Keeping directory documents up-to-date while running
* Acting as a directory cache server
* lots of optimizations that Tor does
* lots of security stuff that Tor does
* pluggable transports
* bridges
* working with no_std, at all.
* being a relay
* disabling unused code
* rate limiting
* API stability
I do not plan to implement full versions of any of those before I
share this code for more comment, though I might do a little. Who
knows?
## Protocol support
Here's a checklist of Tor sub-protocol versions that we currently have
client-side support for:
* [x] `Cons=2` [^1]
* [x] `Desc=2`
* [x] `DirCache=2`
* [x] `FlowCtrl=0`
* [x] `FlowCtrl=1`
* [ ] `HSDir=2`
* [ ] `HSIntro=4` (maybe?)
* [ ] `HSIntro=5`
* [ ] `HSRend=2`
* [x] `Link=4`
* [ ] `Link=5`
* [x] `Microdesc=2` [^1]
* [ ] `Padding=2`
* [x] `Relay=2`
* [ ] `Relay=3` (not sure whether we're compliant with this client-side)
[^1]: If a consensus method before 28 is used, we won't find IPv6 addresses
correctly. All such consensus methods are currently obsolete, though, and
authorities won't negotiate them any more.
There is no relay-side support for anything right now. If there were, we'd
want to support:
* [ ] `Cons=2`
* [ ] `Desc=2`
* [ ] `DirCache=2`
* [ ] `FlowCtrl=0`
* [ ] `FlowCtrl=1`
* [ ] `HSDir=2`
* [ ] `HSIntro=4` (maybe)
* [ ] `HSIntro=5`
* [ ] `HSRend=2`
* [ ] `Link=4`
* [ ] `Link=5`
* [ ] `LinkAuth=3`
* [ ] `Microdesc=2`
* [ ] `Padding=2`
* [ ] `Relay=2`
* [ ] `Relay=3`
We do not ever plan to support these:
* ❌ `Cons=1` (obsolete format)
* ❌ `Desc=1` (obsolete format)
* ❌ `DirCache=1` (no relays still support this)
* ❌ `HSDir=2`
* ❌ `HSIntro=3` (will be obsolete in 2021)
* ❌ `HSRend=1` (will be obsolete in 2021)
* ❌ `LinkAuth=1` (only used by RSA-only relays)
* ❌ `Microdesc=1` (obsolete format)
* ❌ `Padding=1` (deprecated)
We presume that these proposals will be accepted and merged into Tor:
* [ ] 285 (Directory documents should be standardized as utf-8, tor#40131)
* [ ] 315 (Make some directory fields "required", tor#40132)
## What can I do with this?
You can build this all with `cargo build`.
You can run the tests, such as they are, with `cargo test`.
You can make documentation with `cargo doc`. I prefer
`cargo doc --no-deps --document-private-items`, to include documentation for
private members but not for dependencies.
You can try running the demo SOCKS proxy code in `arti` with
`cargo run`. Since rust builds code without optimization by default,
you'll probably want to say `cargo run --release`.
## I want to help. What _should_ I do with this?
Please check `CONTRIBUTING.md` for more dev related information.
More tests would be great.
Parsing more document types would be neat.
* [Compatibility guide](./doc/Compatibility.md)
More documentation examples would be great.
* [Frequently Asked Questions](./doc/FAQ.md)
Improvements or bugfixes to the existing code would be great.
## Why rewrite Tor in Rust?
Improving the look and feel of the documentation would also rock.
Rust is *more secure than C*. Despite our efforts, it's all too simple to
mess up when using a language that does not enforce memory safety. We
estimate that at least half of our tracked security vulnerabilities would
have been impossible in Rust, and many of the others would have been very
unlikely.
I've made a bunch of notes throughout the document in comments with strings
like "XXX" or "TODO".
Rust enables *faster development than C*. Because of Rust's expressiveness
and strong guarantees, we've found that we can be far more efficient and
confident writing code in Rust. We hope that in the long run this will
improve the pace of our software development.
There is a list of features that I wish other crates had in
`WANT_FROM_OTHER_CRATES`.
Arti is *more flexible than our C tor implementation*. Unlike our C `tor`,
which was designed as SOCKS proxy originally, and whose integration features
were later "bolted on", Arti is designed from the ground up to work as a
modular, embeddable library that other applications can use.
Please hold off on opening tickets unless you are pretty sure that you're
reporting a problem I don't know about. :)
Arti is *cleaner than our C tor implementation*. Although we've tried to
develop C tor well, we've learned a lot since we started it back in 2002.
There are lots of places in the current C codebase where complicated
"spaghetti" relationships betwen different pieces of code make our software
needlessly hard to understand and improve.
## Caveat haxxor: what to watch out for
This is a work in progress. It doesn't "do Tor" yet, and what parts
of Tor it does "do" it probably doesn't do securely.
## <a name="status"></a>Current status
I'm learning Rust here as I go along. There are probably aspects of
the language or its ecosystem that I'm getting wrong.
Arti is a work-in-progress. It can connect to the Tor network, bootstrap a
view of the Tor directory, and make anonymized connections over the network.
Almost nothing about this code should be taken as "final" -- I
expect that if anybody wants to make this work for real purposes,
we'll need to refactor and move around a whole bunch of code, add a
bunch of APIs, split crates, merge crates, and so on.
Arti is currently missing some features that are required for good privacy,
and is therefore vulnerable to a lot of attacks that do not yet affect the
Tor network. Therefore, **you should probably not use Arti in production**
if you want any kind of privacy at all.
There are some places where I am deviating from the existing
protocol under the assumption that certain proposals will be
accepted. I'll try to document those.
There are absolutely no guarantees about API stability yet: if you write code
that uses Arti, you should expect that
This code does not try to be indistinguishable from the current Tor
implementation.
## Trying it out today
## Structure
To try to keep dependency relationships reasonable, and to follow
what I imagine to be best practice, I'm splitting this
implementation into a bunch of little crates within a workspace.
Crates that are tor-specific start with "tor-"; others don't.
I expect that the list of crates will have to be reorganized quite a
lot by the time we're done.
The current crates are:
`caret`: A utility for generating enumerations with helpful trait
implementations
`tor-units`: Utilities for wrapping bounded and/or meaningful numeric types.
`tor-llcrypto`: Wrappers and re-exports of cryptographic code that Tor needs in
various ways. Other crates should use this crate, and not actually
use any crypto implementation crates directly. (It's okay to use crates that
define cryptographic traits.)
`tor-rtcompat`: Traits to expose a common interface for asynchronous runtime
code. Currently it supports async-std and tokio.
`tor-rtmock`: Implementations of the traits in `tor-rtcompat` to support
testing.
`tor-bytes`: Byte-by-byte encoder and decoder functions and traits. We use
this to safely parse cells, certs, and other byte-oriented things.
`tor-cert`: Decoding and checking signatures on Tor's ed25519 certificates.
`tor-protover`: Minimal implementation of the Tor subprotocol verisoning
system. Less complete than the one in Tor's current src/rust, but more
simple.
`tor-socksproto`: Implements the server side of the SOCKS protocol, along
with Tor-specific extensions.
Arti can act as a SOCKS proxy that uses the Tor network. (Not a very
secure or anonymous one!) It knows how to download directory
information and how to load it from cache, but it doesn't try to
download more than one directory per run.
`tor-checkable`: Defines traits and types used to represent things that you
can't use until verifying their signatures and checking their timeliness.
To try it out, run the demo program in `arti` as follows. It will open a
SOCKS proxy on port 9150.
`tor-consdiff`: Implements the client side of Tor's consensus-diff algorithm.
% cargo run --release
`tor-netdoc`: Parsing for Tor's network documents. Underdocumented and too
big.
Again, do not use this program yet if you need anonymity, privacy, security,
or stability.
`tor-linkspec`: Traits and types for connecting and extending to Tor relays.
## Helping out
`tor-cell`: Encoding and decoding for Tor cells.
Have a look at our [contributer guidelines](./CONTRIBUTING.md).
`tor-proto`: Functions to work with handshakes, channels, circuits, streams,
and other aspects of the Tor protocol. This crate is NOT ALLOWED to have any
dependencies on specific TLS libraries or specific async environments; those
have to happen at a higher level. (Perhaps this crate should be split into a
cell handling API and a network API. The division point would fairly
logical.)
## Roadmap
`tor-netdir`: Wraps tor-netdoc to expose a "tor network directory" interface.
Doesn't touch the network itself. Right now it only handles microdesc-based
directories.
Thanks to a generous grant from
[Zcash Open Major Grants (ZOMG)](https://zcashomg.org/), we're able to devote
some significant time to Arti in the years 2021-2022. Here is our _rough_
set of plans for what we hope to deliver when.
`tor-chanmgr`: Creates channels as necessary, returning existing channels
when they already exist.
The goal times below are complete imagination, based on broad assumptions about
developer availability. Please don't take them too seriously until we can
get our project manager to sign off on them.
`tor-circmgr`: Creates circuits as requested, returning existing circuits
when they already exist.
* Arti 0.0.1: Minimal Secure Client (Goal: end of October 2021??)
* Target audience: **developers**
* [ ] Guard support
* [ ] Stream Isolation
* [ ] High test coverage
* [ ] Draft APIs for basic usage
* [ ] Code cleanups
* [and more...](https://gitlab.torproject.org/tpo/core/arti/-/milestones/6)
`tor-dirclient`: Downloads directory information over a one-hop circuit.
* Arti 0.1.0: Okay for experimental embedding (Goal: Mid March, 2022??)
* Target audience: **beta testers**
* [ ] Performance: preemptive circuit construction
* [ ] Performance: circuit build timeout inference
* [ ] API support for embedding
* [ ] API support for status reporting
* [ ] Correct timeout behavior
* [and more...](https://gitlab.torproject.org/tpo/core/arti/-/milestones/7)
`tor-dirmgr`: Uses `tor-dirclient` to fetch directory information as needed
to download, cache, and maintain an up-to-date network view. Exposes the
network view as an instance of `tor-netdir::NetDir`.
* Arti 1.0.0: Initial stable release (Goal: Mid September, 2022??)
* Target audience: **initial users**
* [ ] Security audit
* [ ] Stable API
* [ ] Stable CLI
* [ ] Stable configuration format
* [ ] Automatic detection and response of more kinds of network problems
* [ ] More performance work
* [and more...](https://gitlab.torproject.org/tpo/core/arti/-/milestones/8)
`tor-config`: Support for loading and managing configuration files.
* Arti 1.1.0: Anti-censorship features (Goal: End of october, 2022?)
* Target audience: **censored users**
* [ ] Bridges
* [ ] Pluggable transports
* [and more...?](https://gitlab.torproject.org/tpo/core/arti/-/milestones/10)
`tor-client`: A client library that can be used to connect to the Tor network
and make connections.
* Arti 1.2.0: Onion service support (not funded, timeframe TBD)
`arti`: A simple command-line client program that can run as a SOCKS proxy.
* Arti 2.0.0: Feature parity with C tor as a client (not funded, timeframe TBD)
## Intended architecture
* Arti ?.?.?: Relay support
I'm hoping to have small, optional, separable pieces here. For everything
I'm writing, I'm hoping it can go behind a configuration flag.
## How can I help out?
I'm also hoping to focus on everything that _isn't_ the network first. I'd
like as little code as possible to actually read and write to the network,
and as much code as possible to pretend that the network doesn't exist. I
hope this will make everything easier to test.
See [`CONTRIBUTING.md`](./CONTRIBUTING.md) for a few ideas for how to get
started.
## Licence
As appears to be standard practice in the Rust ecosystem, this code is
licensed under either of
This code is licensed under either of
* [Apache License, Version 2.0](http://www.apache.org/licenses/LICENSE-2.0)
* [MIT license](http://opensource.org/licenses/MIT)
......
doc/Architecture.md 0 → 100644
+ 92
− 0
View file @ 99a19ea0
# Arti: Architectural notes and otutline.
## Guidelines
I'm hoping to have small, optional, separable pieces here.
I'd like as little code as possible to actually read and write to the
network, and as much code as possible to pretend that the network doesn't
exist. I hope this will make everything easier to test.
## Structure
To try to keep dependency relationships reasonable, and to follow
what I imagine to be best practice, I'm splitting this
implementation into a bunch of little crates within a workspace.
Crates that are tor-specific start with "tor-"; others don't.
I expect that the list of crates will have to be reorganized quite a
lot by the time we're done.
The current crates are:
`caret`: A utility for generating enumerations with helpful trait
implementations
`tor-units`: Utilities for wrapping bounded and/or meaningful numeric types.
`tor-llcrypto`: Wrappers and re-exports of cryptographic code that Tor needs in
various ways. Other crates should use this crate, and not actually
use any crypto implementation crates directly. (It's okay to use crates that
define cryptographic traits.)
`tor-rtcompat`: Traits to expose a common interface for asynchronous runtime
code. Currently it supports async-std and tokio.
`tor-rtmock`: Implementations of the traits in `tor-rtcompat` to support
testing.
`tor-bytes`: Byte-by-byte encoder and decoder functions and traits. We use
this to safely parse cells, certs, and other byte-oriented things.
`tor-cert`: Decoding and checking signatures on Tor's ed25519 certificates.
`tor-protover`: Minimal implementation of the Tor subprotocol verisoning
system. Less complete than the one in Tor's current src/rust, but more
simple.
`tor-socksproto`: Implements the server side of the SOCKS protocol, along
with Tor-specific extensions.
`tor-checkable`: Defines traits and types used to represent things that you
can't use until verifying their signatures and checking their timeliness.
`tor-consdiff`: Implements the client side of Tor's consensus-diff algorithm.
`tor-netdoc`: Parsing for Tor's network documents. Underdocumented and too
big.
`tor-linkspec`: Traits and types for connecting and extending to Tor relays.
`tor-cell`: Encoding and decoding for Tor cells.
`tor-proto`: Functions to work with handshakes, channels, circuits, streams,
and other aspects of the Tor protocol. This crate is NOT ALLOWED to have any
dependencies on specific TLS libraries or specific async environments; those
have to happen at a higher level. (Perhaps this crate should be split into a
cell handling API and a network API. The division point would fairly
logical.)
`tor-netdir`: Wraps tor-netdoc to expose a "tor network directory" interface.
Doesn't touch the network itself. Right now it only handles microdesc-based
directories.
`tor-chanmgr`: Creates channels as necessary, returning existing channels
when they already exist.
`tor-circmgr`: Creates circuits as requested, returning existing circuits
when they already exist.
`tor-dirclient`: Downloads directory information over a one-hop circuit.
`tor-dirmgr`: Uses `tor-dirclient` to fetch directory information as needed
to download, cache, and maintain an up-to-date network view. Exposes the
network view as an instance of `tor-netdir::NetDir`.
`tor-config`: Support for loading and managing configuration files.
`tor-client`: A client library that can be used to connect to the Tor network
and make connections.
`arti`: A simple command-line client program that can run as a SOCKS proxy.
doc/Compatibility.md 0 → 100644
+ 74
− 0
View file @ 99a19ea0
# Protocol support and compatibility in Arti
Summary:
There is no support in Arti yet for running as a relay, running as a
directory authority, using onion services, or providing onion services.
There is also no anticensorship support. We hope to build these in the
future, but the timeline is not set.
Arti aims for compatibility with all _currently recommended_ Tor protocols.
We have not implemented, and do not plan to implement, obsolete versions of
anything.
## Protocol support
Here's a checklist of Tor sub-protocol versions that we currently have
client-side support for:
* [x] `Cons=2` [^1]
* [x] `Desc=2`
* [x] `DirCache=2`
* [x] `FlowCtrl=0`
* [x] `FlowCtrl=1`
* [ ] `HSDir=2`
* [ ] `HSIntro=4` (maybe?)
* [ ] `HSIntro=5`
* [ ] `HSRend=2`
* [x] `Link=4`
* [ ] `Link=5`
* [x] `Microdesc=2` [^1]
* [ ] `Padding=2`
* [x] `Relay=2`
* [ ] `Relay=3` (not sure whether we're compliant with this client-side)
[^1]: If a consensus method before 28 is used, we won't find IPv6 addresses
correctly. All such consensus methods are currently obsolete, though, and
authorities won't negotiate them any more.
There is no relay-side support for anything right now. If there were, we'd
want to support:
* [ ] `Cons=2`
* [ ] `Desc=2`
* [ ] `DirCache=2`
* [ ] `FlowCtrl=0`
* [ ] `FlowCtrl=1`
* [ ] `HSDir=2`
* [ ] `HSIntro=4` (maybe)
* [ ] `HSIntro=5`
* [ ] `HSRend=2`
* [ ] `Link=4`
* [ ] `Link=5`
* [ ] `LinkAuth=3`
* [ ] `Microdesc=2`
* [ ] `Padding=2`
* [ ] `Relay=2`
* [ ] `Relay=3`
We do not ever plan to support these:
* `Cons=1` (obsolete format)
* `Desc=1` (obsolete format)
* `DirCache=1` (no relays still support this)
* `HSDir=2`
* `HSIntro=3` (will be obsolete in 2021)
* `HSRend=1` (will be obsolete in 2021)
* `LinkAuth=1` (only used by RSA-only relays)
* `Microdesc=1` (obsolete format)
* `Padding=1` (deprecated)
We presume that these proposals will be accepted and merged into Tor:
* [ ] 285 (Directory documents should be standardized as utf-8, tor#40131)
* [ ] 315 (Make some directory fields "required", tor#40132)
doc/FAQ.md 0 → 100644
+ 21
− 0
View file @ 99a19ea0
# Arti: Frequently Asked Questions
## What is Arti?
Arti is a project to rewrite the entire codebase of the
[Tor anonymity network](https://torproject.org) in
[Rust](https://rustlang.org/).
Arti is also the name of the software produced by this project.
## What is the status of Arti today?
As of mid-June 2021: Arti ready for developers to play around with, but it is
not yet ready for general use. Arti can run as a simple Tor client and
send anonymized traffic over the network; it has no support for running as a
relay or for using onion services.
## Should I use Arti?
As of mid-June 2021: probably not yet.
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment