It's a compute market.

You can buy clusters with 1000+ H100s with InfiniBand at volume prices and then sell back any portion of it whenever you'd like. Other providers prevent you from reselling and recovering funds.

We made it a CLI command. First you can buy any shape of compute you'd like.

# buy 64 h100s         -n 64
# for 24 hours         -d "24h" 
# starting tomorrow    -s "tomorrow at 9am"
sf buy -n 64 -d "24h" -s "tomorrow at 9am"

You can also buy and get access to nodes right away:

# buy 8 h100s                -n 8
# for 1 hour                 -d "1hr"
# at $2.50 per GPU per hour  -p "2.50"
sf buy -d "1hr" -n 8 -p "2.50"

In this example, you're willing to pay up to $2.50 per GPU per hour. This means the maximum total cost would be 2.50 x 8 = $20 per hour, but you may end up paying less depending on market conditions and available supply.

And you can sell:

# sell 1024 H100s        -n 128
# for 12 hours           -d "12h"
# starting at 7pm        -s "tonight at 7pm"
# at $3.50/gpu/hour      -p '3.50'
# from contract cont_123 -c cont_123
sf sell -c cont_123 -p '3.50' -n 128 -d '12h' -s 'tonight at 7pm'

An important thing to note - these are orders, not reservations. By placing an order, you're signaling to the market that you're willing to buy or sell a block of compute at a certain price. You haven't actually bought or sold anything until your order gets filled.

If you place a buy order, your order won't be filled unless there's a corresponding sell order for the same amount of compute at the same price or lower.

Similarly, if you place a sell order, your order won't be filled unless there's a corresponding buy order for the same amount of compute at the same price or higher.

If you place an order and it can't immediately be filled, it will sit in the market until you cancel it, it expires, or it gets filled. You can see all of the outstanding orders with sf orders list --public or by visiting the blackbook.

To place an order on SF Compute, you'll need to sign up on the website, sign a services agreement, and fund your account. If you'd like to talk to us before you do so, you can reach us at contact@sfcompute.com.

Then, you can download the command line tool.

curl -fsSL https://sfcompute.com/cli/install | bash

Next, login via the CLI.

sf login # this will open a browser window

Let's place an order for 1 day on 1024 H100s with InfiniBand (128 nodes), starting tomorrow at 10am, for $5/hour.

sf buy -t h100i -n 1024 -p '5' -s 'tomorrow at 10am' -d '1d'
# ... it will confirm with the price before executing

Hmm, $5 an hour isn't crazy for a last minute 1024 fully-interconnected cluster for only a day, but that's way little higher than we want. Instead, let's set a max price. If someone offers our price or lower, we'll get the cluster, otherwise the order is canceled.

We'll set a low price, like $1.5 / hr, and see if we can get lucky. Maybe someone's code isn't ready yet, and they need to firesale their cluster to recoup costs.

sf buy -t h100i -n 1024 -s 'tomorrow at 10am' -d '1d' -p '1.5'
# ... it will confirm before executing

You can check the status of orders with orders list.

sf orders list
 
BUY ord_1a2b3c4d5e 
  Type: h100i 
  Quantity: 128
  Start: 2024-06-15 10:00:00 UTC
  Duration: 1d
  Price: $36,864
  Status: Pending
 
# ...

If your order has been filled and has started, you'll be provisioned VMs. You can see your instances with:

sf instances list
 
main-1   h100i   34.102.156.1   angel-island   cont_aGVsbG93b3Js
main-2   h100i   34.102.156.2   angel-island   cont_aGVsbG93b3Js
main-3   h100i   34.102.156.3   angel-island   cont_aGVsbG93b3Js
main-4   h100i   34.102.156.4   angel-island   cont_aGVsbG93b3Js
 
# ...

There are two options for setting up SSH keys:

This method automatically generates and sets up a public key for you. This only needs to be done once per device you're connecting from. Run the following command:

sf ssh --init

1. Generate an Ed25519 key (recommended):

ssh-keygen -t ed25519 -C "your_email@example.com"

Alternatively, you can create an RSA key:

ssh-keygen -t rsa -b 4096 -C "your_email@example.com"

2. Add your public key to SF Compute:

sf ssh --add ~/.ssh/id_ed25519.pub

After setting up your SSH keys, you can connect to your instances using sf ssh:

1. Wait until your machine has a "healthy" status under sf instances list
2. Connect using either the instance ID or machine name:

   sf ssh instance_id

   or

   sf ssh machine_name

Replace instance_id or machine_name with the appropriate value. You will see the name of your machine once you have logged into your node. The default user for all VMs is ubuntu.

For example, when your instance is running, you will see an output like the following:

sf instances list
 
inst_1pE5Wprlr6T1C72onaj9kzdsymqrHc8IJmRnP  h100i  38.104.174.59:2219  healthy

You can ssh into your machine like so:

sf ssh inst_1pE5Wprlr6T1C72onaj9kzdsymqrHc8IJmRnP

When your order is filled, you're given a "contract" for some amount of compute for some period of time. You can see the contracts you own with contracts list.

sf contracts list
 
cont_aGVsbG93b3Js   h100i   64  2024-06-15 10:00:00 UTC  2024-06-16 10:00:00 UTC
cont_bXlwYXNzd29y   h100i   8   2024-06-16 12:00:00 UTC  2024-06-17 12:00:00 UTC

If you have nodes, you can sell them back at any point. Let's sell back next weeks nodes from one of the contracts we own.

sf sell -c cont_aGVsbG93b3Js -p '2.85' -n 64 -d '1w' -s 'tomorrow at 10am'