Minio简介
MinIO是一款基于Apache License v2.0开源协议的分布式文件系统(或者叫对象存储服务),可以做为云存储的解决方案用来保存海量的图片、视频、文档等。由于采用Golang实现,服务端可以工作在Windows、Linux、 OS X和FreeBSD上。配置简单,基本是复制可执行程序,单行命令就可以运行起来。
MinIO兼容亚马逊S3(Simple Storage Service,简单存储服务)云存储服务接口,非常适合于存储大容量非结构化的数据,例如图片、视频、日志文件、备份数据和容器/虚拟机镜像等,而且每个对象文件可以是任意大小,从几kb到最大5T不等。
基本概念
- bucket(桶) :类似文件系统的目录(文件夹);
- Object : 类似文件系统的文件;
- Keys :类似文件名;
- MINIO_ACCESS_KEY:访问key,类似账号;
- MINIO_SECRET_KEY:秘钥,类似密码。
MinIO特点
- 高性能:作为高性能对象存储,在标准硬件条件下它能达到55GB/s的读、35GB/s的写速率;
- 可扩容:不同MinIO集群可以组成联邦,并形成一个全局的命名空间,并跨越多个数据中心;
- SDK支持: 基于Minio轻量的特点,它得到类似Java、Python或Go等语言的sdk支持;
- 支持纠删码:MinIO使用纠删码、Checksum来防止硬件错误和静默数据污染。在最高冗余度配置下,即使丢失1/2的磁盘也能恢复数据;
与FastDFS对比
- FastDFS是阿里余庆做的个人项目,也是一款开源高性能的分布式文件系统,适合小规模文件数据存储,默认不提供UI界面,安装部署(运维)复杂,很难达到以G为单位的每秒读写速度,没有完备的官方文档,环境搭建较为复杂;
- MinIO是由MinIO.Inc运营的开源项目,号称世界上速度最快的对象存储服务器,并且社区活跃度高,标准硬件条件下它能达到55GB/s的读、35GB/s的写速率,而且MinIO部署自带管理界面,不需要额外安装;MinIO提供了所有主流开发语言的SDK,并且兼容亚马逊S3云存储服务接口,在MinIO中一个对象文件可以是任意大小,从几KB到最大的5T不等;最后它提供了与k8s、etcd、docker等容器技术深度集成方案,可以说就是为云原生而生的。(缺点,不支持动态增加节点)
MinIO快速使用
1、下载MinIO安装包
下载地址:https://www.minio.org.cn/download.shtml#/windows
备用地址:https://dl.minio.io/server/minio/release/windows-amd64/minio.exe
2、创建数据存储文件夹data
3、在minio.exe文件夹的路径处输入cmd进入命令行界面,启动MinIO服务:
minio.exe server .\data
命令行输出:
Formatting 1st pool, 1 set(s), 1 drives per set.
WARNING: Host local has more than 0 drives of set. A host failure will result in data becoming unavailable.
WARNING: Detected default credentials 'minioadmin:minioadmin', we recommend that you change these values with 'MINIO_ROOT_USER' and 'MINIO_ROOT_PASSWORD' environment variables
MinIO Object Storage Server
Copyright: 2015-2023 MinIO, Inc.
License: GNU AGPLv3 <https://www.gnu.org/licenses/agpl-3.0.html>
Version: RELEASE.2023-08-04T17-40-21Z (go1.19.12 windows/amd64)
Status: 1 Online, 0 Offline.
S3-API: http://192.168.10.41:9000 http://192.168.140.1:9000 http://192.168.46.1:9000 http://127.0.0.1:9000
RootUser: minioadmin
RootPass: minioadmin
Console: http://192.168.10.41:53081 http://192.168.140.1:53081 http://192.168.46.1:53081 http://127.0.0.1:53081
RootUser: minioadmin
RootPass: minioadmin
Command-line: https://min.io/docs/minio/linux/reference/minio-mc.html#quickstart
$ mc.exe alias set myminio http://192.168.10.41:9000 minioadmin minioadmin
Documentation: https://min.io/docs/minio/linux/index.html
Warning: The standard parity is set to 0. This can lead to data loss.
IAM refresh took 19.79s
4、浏览器访问http://127.0.0.1:53081
,输入账号minioadmin,密码minioadmin即可登录。
5、创建桶,可将桶看作文件夹,这里我创建了一个fangfile桶,设置Access Policy为public。
6、创建Access Key,进入Access Keys页面,点Create access key,记录Access Key和Secret Key。
Spring Boot集成MinIO
1、创建Spring Boot应用
2、pom文件添加相关依赖
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>2.2.1.RELEASE</version>
<relativePath/> <!-- lookup parent from repository -->
</parent>
<groupId>com.fang</groupId>
<artifactId>minio-demo</artifactId>
<version>0.0.1-SNAPSHOT</version>
<name>minio-demo</name>
<description>Demo project for Spring Boot</description>
<properties>
<java.version>1.8</java.version>
</properties>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.projectlombok</groupId>
<artifactId>lombok</artifactId>
<optional>true</optional>
</dependency>
<dependency>
<groupId>io.minio</groupId>
<artifactId>minio</artifactId>
<version>7.1.0</version>
</dependency>
<dependency>
<groupId>com.squareup.okhttp3</groupId>
<artifactId>okhttp</artifactId>
<version>3.14.9</version>
</dependency>
<dependency>
<groupId>com.github.xiaoymin</groupId>
<artifactId>knife4j-spring-boot-starter</artifactId>
<version>2.0.5</version>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
</dependencies>
3、整合Swagger,编写Swagger配置类
package com.fang.miniodemo.config;
import com.github.xiaoymin.knife4j.spring.annotations.EnableKnife4j;
import org.springframework.context.annotation.Configuration;
import springfox.documentation.builders.ApiInfoBuilder;
import springfox.documentation.builders.PathSelectors;
import springfox.documentation.builders.RequestHandlerSelectors;
import springfox.documentation.service.ApiInfo;
import springfox.documentation.service.Contact;
import springfox.documentation.spi.DocumentationType;
import springfox.documentation.spring.web.plugins.Docket;
import springfox.documentation.swagger2.annotations.EnableSwagger2;
@Configuration
@EnableSwagger2
@EnableKnife4j
public class SwaggerConfig {
public Docket webApiConfig() {
return new Docket(DocumentationType.SWAGGER_2)
.groupName("webApi")
.apiInfo(webApiInfo())
.select()
.apis(RequestHandlerSelectors.basePackage("com.fang.controller"))
.paths(PathSelectors.any())
.build();
}
private ApiInfo webApiInfo() {
return new ApiInfoBuilder()
.title("网站-API文档")
.description("微服务接口定义")
.version("1.0")
.contact(new Contact("方伟俊", "", "568036792@qq.com"))
.build();
}
}
4、编写MinIO属性配置类
package com.fang.miniodemo.config;
import lombok.Data;
import org.springframework.boot.context.properties.ConfigurationProperties;
import org.springframework.stereotype.Component;
import java.io.Serializable;
@Data
@Component
@ConfigurationProperties(prefix = "minio")
public class MinIOConfigProperties implements Serializable {
private String accessKey;
private String secretKey;
private String bucket;
private String endpoint;
private String readPath;
}
5、编写MinIO配置类,注册MinioClient客户端Bean对象
package com.fang.miniodemo.config;
import io.minio.MinioClient;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
@Configuration
public class MinIOConfig {
@Autowired
private MinIOConfigProperties minIOConfigProperties;
@Bean
public MinioClient buildMinioClient() {
return MinioClient.builder()
.credentials(minIOConfigProperties.getAccessKey(), minIOConfigProperties.getSecretKey())
.endpoint(minIOConfigProperties.getEndpoint())
.build();
}
}
6、在application.properties文件配置minio自定义属性
minio.accessKey=XM5jigwjjwMMWAb9tHEf
minio.secretKey=rZyoUixUUtVDDl8O9TSvszFXl1ODbeYL9t3gJOMe
minio.bucket=fangfile
minio.endpoint=http://192.168.10.41:9000
minio.readPath=http://192.168.10.41:9000
7、编写minio业务接口
package com.fang.miniodemo.service;
import java.io.InputStream;
public interface FileStorageService {
String uploadImgFile(String prefix, String filename, InputStream inputStream);
}
业务接口实现类:
package com.fang.miniodemo.service.impl;
import com.fang.miniodemo.config.MinIOConfigProperties;
import com.fang.miniodemo.service.FileStorageService;
import io.minio.MinioClient;
import io.minio.PutObjectArgs;
import lombok.extern.slf4j.Slf4j;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Service;
import org.springframework.util.StringUtils;
import java.io.InputStream;
import java.text.SimpleDateFormat;
import java.util.Date;
@Slf4j
@Service
public class MinIOFileStorageService implements FileStorageService {
@Autowired
private MinioClient minioClient;
@Autowired
private MinIOConfigProperties minIOConfigProperties;
private final static String separator = "/"; //文件夹分隔符
/**
* 构建文件的绝对路径
*
* @param dirPath 文件路径
* @param filename 文件名 yyyy/mm/dd/file.jpg
* @return /test
*/
public String builderFilePath(String dirPath, String filename) {
StringBuilder stringBuilder = new StringBuilder(50);
if (!StringUtils.isEmpty(dirPath)) {
stringBuilder.append(dirPath).append(separator);
}
SimpleDateFormat sdf = new SimpleDateFormat("yyyy/MM/dd");
String todayStr = sdf.format(new Date());
stringBuilder.append(todayStr).append(separator);
stringBuilder.append(filename);
return stringBuilder.toString();
}
@Override
public String uploadImgFile(String prefix, String filename, InputStream inputStream) {
String filePath = builderFilePath(prefix, filename);
try {
PutObjectArgs putObjectArgs = PutObjectArgs.builder()
.object(filePath)
.contentType("image/jpg")
.bucket(minIOConfigProperties.getBucket())
.stream(inputStream, inputStream.available(), -1)
.build();
minioClient.putObject(putObjectArgs);
StringBuilder urlPath = new StringBuilder(minIOConfigProperties.getReadPath());
urlPath.append(separator + minIOConfigProperties.getBucket());
urlPath.append(separator);
urlPath.append(filePath);
return urlPath.toString();
} catch (Exception ex) {
log.error("minio put file error: ", ex);
throw new RuntimeException("上传文件失败");
}
}
}
8、编写统一结果处理类
package com.fang.miniodemo.dto;
import lombok.Data;
@Data
public class Result<T> {
private Integer code; //响应状态码
private String msg; //响应消息
private T data; //响应数据
/**
* 处理成功的返回结果
*
* @param data 数据
* @param <T>
* @return R<T>
*/
public static <T> Result<T> success(T data, String msg) {
Result<T> r = new Result<>();
r.setCode(200);
r.setData(data);
r.setMsg(msg);
return r;
}
/**
* 处理成功的返回结果
*
* @param msg 错误信息
* @param <T>
* @return R<T>
*/
public static <T> Result<T> error(String msg) {
Result<T> r = new Result<>();
r.setCode(503);
r.setMsg(msg);
return r;
}
}
9、编写Controller类
package com.fang.miniodemo.controller;
import com.fang.miniodemo.dto.Result;
import com.fang.miniodemo.service.FileStorageService;
import io.swagger.annotations.Api;
import io.swagger.annotations.ApiOperation;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;
import org.springframework.web.multipart.MultipartFile;
import java.io.IOException;
import java.io.InputStream;
import java.util.UUID;
@RestController
@RequestMapping("/minio")
@Api(tags = "minio相关接口")
public class MinioController {
@Autowired
private FileStorageService fileStorageService;
/**
* 上传图片到minio
*
* @param file
* @return
*/
@PostMapping("upload")
@ApiOperation(value = "图片上传接口")
public Result uploadFile(MultipartFile file) throws IOException {
try {
// 获取文件名称
String fileName = file.getOriginalFilename();
/*解决多次上传同名文件覆盖问题*/
// 在文件名称里面添加随机唯一的值
String uuid = UUID.randomUUID().toString().replaceAll("-", "");
fileName = uuid + fileName;
// 获取文件输入流
InputStream is = file.getInputStream();
String imgUrl = fileStorageService.uploadImgFile("img", fileName, is);
return Result.success(imgUrl, "上传成功");
} catch (IOException e) {
e.printStackTrace();
return Result.error("上传失败");
}
}
}
10、启动项目,通过Swagger调试界面上传文件测试,上传后可以看到返回的文件路径,可以看到MinIO界面已上传的文件。
Q&A
1、官网教程Maven引用MinIO包后,启动时报错:程序包io.minio不存在
<dependency>
<groupId>io.minio</groupId>
<artifactId>minio</artifactId>
<version>8.5.4</version>
</dependency>
解决:首先到maven仓库找到对应的jar包并下载到本地,maven官网
这里下载的是7.1.0版本,在命令行执行以下命令:
mvn install:install-file -Dfile=minio-7.1.0.jar -DgroupId=io.minio -DartifactId=minio -Dversion=7.1.0 -Dpackaging=jar
-Dfile:从仓库下载的jar包的存放路径
-DgroupId:对应依赖的groupId
-DartifactId:对应依赖的artifactId
-Dversion:对应依赖的version
-Dpackaging:对应文件类型 jar
安装成功后,将Spring Boot工程里minio的依赖改成7.1.0。
2、解决完以上问题后,启动时报错:MinIOConfig.java: 无法访问okhttp3.HttpUrl,找不到okhttp3.HttpUrl的类文件
解决:添加以下依赖:
<dependency>
<groupId>com.squareup.okhttp3</groupId>
<artifactId>okhttp</artifactId>
<version>3.14.9</version>
</dependency>
3、启动时okhttp3报error in opening zip file错误
解决:删除maven本地仓库的jar包,然后idea中项目刷新,让maven重新下载依赖。