How to list files uploaded to Web3.Storage

In this how-to guide, you'll learn about the different ways that you can list the files that you've uploaded to Web3.Storage. Once you've stored some files using Web3.Storage, you'll want to see a list of what you've uplodaded. There are two ways you can do this:

Using the Web3.Storage website

You can see a list of everything you've uploaded to Web3.Storage on the Files page on the Web3.Storage website. If you don't need to work with this list programmatically, using the website may be a simpler choice.

This Files page provides a convenient overview of your stored data, including links to view your files in your browser via an IPFS gateway and information about how the data is being stored on the decentralized storage networks that Web3.Storage uses under the hood.

Using the Web3.Storage client

To easily integrate Web3.Storage programmatically in your apps or services, you can also access a listing of your uploads from your code using the Web3.Storage client. In the example below, this guide walks through how to use the JavaScript client library to fetch a complete listing of all the data you've uploaded using Web3.Storage.

Installing the client

In your JavaScript project, add the web3.storage package to your dependencies:

npm install web3.storage

Creating a client instance

To create a Web3Storage client object, we need to pass an access token into the constructor:

import { Web3Storage } from 'web3.storage'

function getAccessToken () {
  // If you're just testing, you can paste in a token
  // and uncomment the following line:
  // return 'paste-your-token-here'

  // In a real app, it's better to read an access token from an
  // environement variable or other configuration that's kept outside of
  // your code base. For this to work, you need to set the
  // WEB3STORAGE_TOKEN environment variable before you run your code.
  return process.env.WEB3STORAGE_TOKEN
}

function makeStorageClient () {
  return new Web3Storage({ token: getAccessToken() })
}

Listing your uploads

The Web3Storage client object's list method returns an async iterable that can be used with the for await syntax to read information about each upload as soon as it's received over the network.

Here's an example that logs details about each upload to the console:

async function listUploads () {
  const client = makeStorageClient()
  for await (const upload of client.list()) {
    console.log(`${upload.name} - cid: ${upload.cid} - size: ${upload.dagSize}`)
  }
}

Each Upload object will look something like this:

{
  "name": "cat pics",
  "cid": "bafkreifzjut3te2nhyekklss27nh3k72ysco7y32koao5eei66wof36n5e",
  "created": "2021-07-14T19:27:14.934572Z",
  "dagSize": 101,
  "pins": [{
    "status": "Pinned"
  }],
  "deals": [{
    "dealId": 12345,
    "storageProvider": "f099",
    "status": "Active",
    "pieceCid": "bafkreifzjut3te2nhyekklss27nh3k72ysco7y32koao5eei66wof36n5e",
    "dataCid": "bafkreifzjut3te2nhyekklss27nh3k72ysco7y32koao5eei66wof36n5e",
    "dataModelSelector": "Links/0/Links",
    "activation": "2021-07-14T19:27:14.934572Z",
    "created": "2021-07-14T19:27:14.934572Z",
    "updated": "2021-07-14T19:27:14.934572Z"
  }]
}

What do all those fields mean? Here's a summary:

• name contains a descriptive name for the upload. If no name was provided at the time of upload, the name field will contain an automatically generated name that includes a creation timestamp.
• cid contains the IPFS Content Identifier (CID) that identifies the uploaded data. This CID can be used to retrieve the uploaded files or get more detailed status information.
• created contains an ISO-8601 datetime string indicating when the content was first uploaded to Web3.Storage.
• dagSize contains the size in bytes of the Directed Acyclic Graph (DAG) that contains all the uploaded content. This is the size of the data that is transferred over the network to Web3.Storage during upload, and is slightly larger than the total size of the files on disk.
• pins contains an array of objects describing the IPFS nodes that have pinned the data, making it available for fast retrieval using the IPFS network.
• deals contains an array of objects describing the Filecoin storage providers that have made storage deals. These storage providers have committed to storing the data for an agreed period of time.

Listing a subset of uploads

By default, the list method returns information about all uploads made using your Web3.Storage account. You can optionally restrict the listing in two ways:

• Only contain entries that were uploaded before a given timestamp.
• Limit the total number of returned entries.

Here's an example of fetching the first 10 uploads made on the previous day:

async function listWithLimits () {
  const client = makeStorageClient()

  // get today's date and subtract 1 day
  const d = new Date()
  d.setDate(d.getDate() - 1)

  // the list method's before parameter accepts an ISO formatted string
  const before = d.toISOString()

  // limit to ten results
  const maxResults = 10

  for await (const upload of client.list({ before, maxResults })) {
    console.log(upload)
  }
}