docker volume对接minio s3存储

启动minio

root@master2:~# docker run -p 9000:9000 minio/minio server /opt/data
Endpoint:  http://172.17.0.2:9000  http://127.0.0.1:9000

Browser Access:
   http://172.17.0.2:9000  http://127.0.0.1:9000

Object API (Amazon S3 compatible):
   Go:         https://docs.min.io/docs/golang-client-quickstart-guide
   Java:       https://docs.min.io/docs/java-client-quickstart-guide
   Python:     https://docs.min.io/docs/python-client-quickstart-guide
   JavaScript: https://docs.min.io/docs/javascript-client-quickstart-guide
   .NET:       https://docs.min.io/docs/dotnet-client-quickstart-guide
Detected default credentials 'minioadmin:minioadmin', please change the credentials immediately using 'MINIO_ACCESS_KEY' and 'MINIO_SECRET_KEY'

*注：因docker版本的不同，16和18版本略有区别，区别在于，16版本需要部署rexray而18版本不需要，部署了反而会发生冲突

安装rexray/s3fs plugin

安装 rexray

安装

root@master2:~# curl -sSL https://rexray.io/install | sh
Selecting previously unselected package rexray.
(Reading database ... 75879 files and directories currently installed.)
Preparing to unpack rexray_0.11.4-1_amd64.deb ...
Unpacking rexray (0.11.4-1) ...
Setting up rexray (0.11.4-1) ...

rexray has been installed to /usr/bin/rexray

REX-Ray
-------
Binary: /usr/bin/rexray
Flavor: client+agent+controller
SemVer: 0.11.4
OsArch: Linux-x86_64
Commit: e7414eaa971b27977d2283f2882825393493179d
Formed: Tue, 15 Jan 2019 11:03:57 EST

配置

新建/etc/rexray/config.yml:

libstorage:
  service: s3fs
s3fs:
  accessKey: #minio用户名
  secretKey: #minio密码
  endpoint: http://10.x.x.x:9000   #minio地址
  region: us-east-1
  disablePathStyle: false 
  options:
  - url=http://10.x.x.x:9000  #minio地址
  - use_path_request_style
  - nonempty

重启rexray

$ sudo rexray service start

安装rexray/s3fs plugin

docker plugin install rexray/s3fs \
  S3FS_OPTIONS="allow_other,use_path_request_style,nonempty,url=http://10.x.x.x:9000" \
  S3FS_ENDPOINT="http://10.x.x.x:9000" \
  S3FS_ACCESSKEY="#minio用户名" \
  S3FS_SECRETKEY="#minio密码" \
  S3FS_REGION="us-east-1"

输出Do you grant the above permissions? [y/N]时，输入y

Plugin "rexray/s3fs" is requesting the following privileges:
 - network: [host]
 - mount: [/dev]
 - allow-all-devices: [true]
 - capabilities: [CAP_SYS_ADMIN]
Do you grant the above permissions? [y/N] y
latest: Pulling from rexray/s3fs
5fa993fcb5a9: Downloading [===>                                               ]  7.515MB/99.62MB

未安装 rexray 会出错

Plugin "rexray/s3fs" is requesting the following privileges:
- network: [host]
- mount: [/dev]
- allow-all-devices: [true]
- capabilities: [CAP_SYS_ADMIN]
Do you grant the above permissions? [y/N] y
 latest: Pulling from rexray/s3fs
 5fa993fcb5a9: Download complete 
 Digest: sha256:8f1ededd08deb86093ece2c80defa263c9aeb3eadfead6b17fe7bd767e417df9
Status: Downloaded newer image for rexray/s3fs:latest
 Error response from daemon: dial unix /run/docker/plugins/649f200b3147cd4a298e9408f5d0b1c96b8bdd0cc1d7203f59aa5dbd032e16e4/rexray.sock: connect: no such file or directory

安装 rexray 后，再次安装plugin

root@master2:~# docker plugin install rexray/s3fs   S3FS_OPTIONS="allow\_other,use\_path\_request\_style,nonempty,url= http://172.17.0.2:9000"   S3FS_ENDPOINT=" http://172.17.0.2:9000"   S3FS_ACCESSKEY="minioadmin"   S3FS_SECRETKEY="minioadmin"
Plugin "rexray/s3fs" is requesting the following privileges:
- network: [host]
- mount: [/dev]
- allow-all-devices: [true]
- capabilities: [CAP_SYS_ADMIN]
Do you grant the above permissions? [y/N] y
Error response from daemon: plugin rexray/s3fs:latest already exists

使用

查看rexray信息

$ rexray env

查看rexray volume

16
rexray volume ls
或
18
docker volume ls

创建volume

16
sudo rexray volume new testbucket
或
18
sudo docker volume create testbucket

挂载给docker容器

16
docker run -it --volume-driver=rexray -v testbucket:/opt/s3 busybox sh
或
18
docker run -it --volume-driver=rexray/s3fs -v testbucket:/opt/s3 busybox sh

至此该docker volume已经可以挂载给容器使用了，兼容s3模式，该volume对应minio的bucket

reference

REX-Ray with Minio · GitHub
REX-Ray with Minio · GitHub
rexray/s3fs Docker Plugin Install with Minio · GitHub
Troubleshooting - REX-Ray