Set up object storage
Access Key and Secret Key
If your cloud service provider supports configuring bucket access policy for virtual machines, and achieve access to object storage without credentials (like AWS IAM), you can omit those keys during juicefs auth
or juicefs mount
(provide empty value), see juicefs auth
for details.
For users that prefer ease of use, we recommend granting full read/write and CreateBucket
permission to the API keys, so that JuiceFS Client can help you create the bucket on the first successful mount.
But for environments that comes with strict security policies, our minimum permission requirements are GetObject
, PutObject
and DeleteObject
. You can restrict access to the corresponding bucket, under the file system prefix path (default to juicefs-<VOL_NAME>
). When running with minimum permissions, JuiceFS will not be able to create object storage bucket for you, create them manually in advance.
- Since JuiceFS clients runs background jobs, like compaction, even if it's mounted with a read-only token, client still needs
PutObject
、DeleteObject
permission to run these background jobs. If you'd like read-only clients to be truly read-only against object storage, disable its background job in client access token. ListObjects
is required for importing and replication.
Common object storage
Amazon S3
Refer to "AWS security credentials". Moreover, if you have used an IAM role to grant permissions to applications running on Amazon EC2 instances, you can omit credentials during juicefs mount
.
A bucket policy example:
{
"Version": "2012-10-17",
"Statement": [
{
"Action": [
"s3:DeleteObject",
"s3:GetObject",
"s3:HeadObject",
"s3:PutObject"
],
"Effect": "Allow",
"Resource": [
"arn:aws:s3:::juicefs-example/example/*"
]
}
]
}
Google Cloud Storage
First, you should create a project in the console of Google Cloud Platform, remember your Project ID:
Download and install Cloud SDK:
curl https://sdk.cloud.google.com | bash
Run the following command after installation:
gcloud auth application-default login
Congratulation, you have done the authentication job that would be executed only once.
Finally, you could run juicefs mount
to mount your JuiceFS file system, the Project ID will be requested (you could set it as GOOGLE_CLOUD_PROJECT
in environment variable).
When you mount a file system with sudo
, you also should run gcloud auth
with sudo
. Otherwise, the JuiceFS may not load the credential.
If JuiceFS is used inside Compute Engine, it's recommended to grant the virtual machines full access to Storage API.
Azure Blob Storage
Currently, service is only available at Microsoft Azure Chinese Region, contact us and other regions can be supported.
When the JuiceFS use the Azure Blob Storage as the underlying storage, you should create a storage account. Find Storage Accounts from the navigation of left panel.
Create a new account in Storage accounts, the name will be requested at mounting the JuiceFS file system, the account kind should be "Blob storage".
Enter the Access key from your storage account, there're two keys available.
Backblaze B2
Create an application key with read and write permission on Application Keys.
The master application key is required to create a bucket by JuiceFS. It's recommended to create a bucket manually, using a name like juicefs-NAME
, then create an application key with read-write access for JuiceFS.
IBM Cloud Object Storage
It requires API Key and Resource Instance ID to access Cloud Object Storage, refer to Retrieving your instance ID.
DigitalOcean Spaces
Refer to How To Create a DigitalOcean Space and API Key.
Wasabi
Refer to Creating a Root Access Key and Secret Key.
Alibaba Cloud OSS
Obtain Access Key in the object storage console:
Create a key for JuiceFS mount:
A bucket policy example:
{
"Statement": [
{
"Action": [
"oss:DeleteObject",
"oss:GetObject",
"oss:HeadObject",
"oss:PutObject"
],
"Effect": "Allow",
"Resource": [
"acs:oss:*:*:juicefs-example/example/*"
]
}
],
"Version": "1"
}
Tencent Cloud COS
When using Tencent Cloud COS, mounting JuiceFS requires a Tencent APPID in addition, so we recommend fill in the APPID into Bucket when creating the file system, using format {bucket}-{APPID}
. If you didn't specify an APPID when creating the file system, JuiceFS will ask for APPID interactively during mount. Moreover, you can specify APPID in juicefs auth
, by the --bucket
parameter, using the same format {bucket}-{APPID}
.
APPID is in the Account Info.
Secret ID and Secret Key are managed in the API Key Management, you need to create a pair if it's empty.
A bucket policy example:
{
"Statement": [
{
"Effect": "Allow",
"Action": [
"cos:DeleteObject",
"cos:GetObject",
"cos:HeadObject",
"cos:PutObject"
],
"Resource": [
"qcs::cos:ap-guangzhou:uid/1250000000:juicefs-example-1250000000/example/*"
]
}
],
"Version": "2.0"
}
Huawei Cloud OBS
Refer to How Do I Manage Access Keys?.
Baidu Cloud BOS
Login the Baidu Cloud Console, enter the Security Authentication in the dropdown menu of the account at the right-upper corner of the page.
Kingsoft Cloud KS3
Refer to User Access Key Management.
QingCloud QingStor
Login the Qingcloud Console, you'll find Access Keys in the dropdown menu of your account at the right corner.
Qiniu Kodo
Refer to How to get Access Key and Secret Key.
UCloud US3
Login the UCloud console, you'll find your API key in UAPI in the Monitoring management of Product and service.
Ceph
Ceph provides two sets of APIs: RADOS and RGW. RADOS is the underlying protocol provided by Ceph, while RGW is a S3 gateway, exposing standard S3 APIs. Connecting via RADOS is recommended as it bypasses RGW and achieves better latency. If you decide to use Ceph via S3, use it like any other S3 object storage services.
If the RADOS client protocol is used, JuiceFS uses librados2
, which supports Ceph >= 12.2. You'll need to provide the cluster name (e.g. ceph
), and a user name (e.g. client.admin
).
Other Object Storage
When creating object storage in the cloud console, choose object storage type and region in Object Storage Region
. If the desired one aren't listed, try selecting a nearby platform area, then fill out the complete endpoint in the Bucket field.
For example, if your MinIO deployment is in a data center in Singapore, and you want JuiceFS cloud services to use it as an underlying storage. You won't find MinIO in our list of object storage services so in this case, select the S3 region that's closest to your DC, check the Advanced Options
, and then enter the complete object storage endpoint in the Bucket field.
All S3 API-compliant object storage, such as Cloudflare R2, can use this method to access.