Spring Boot 2集成Minio

Posted by Kaka Blog on August 10, 2023

Minio简介

MinIO是一款基于Apache License v2.0开源协议的分布式文件系统(或者叫对象存储服务),可以做为云存储的解决方案用来保存海量的图片、视频、文档等。由于采用Golang实现,服务端可以工作在Windows、Linux、 OS X和FreeBSD上。配置简单,基本是复制可执行程序,单行命令就可以运行起来。

MinIO兼容亚马逊S3(Simple Storage Service,简单存储服务)云存储服务接口,非常适合于存储大容量非结构化的数据,例如图片、视频、日志文件、备份数据和容器/虚拟机镜像等,而且每个对象文件可以是任意大小,从几kb到最大5T不等。

基本概念

  • bucket(桶) :类似文件系统的目录(文件夹);
  • Object : 类似文件系统的文件;
  • Keys :类似文件名;
  • MINIO_ACCESS_KEY:访问key,类似账号;
  • MINIO_SECRET_KEY:秘钥,类似密码。

MinIO特点

  • 高性能:作为高性能对象存储,在标准硬件条件下它能达到55GB/s的读、35GB/s的写速率;
  • 可扩容:不同MinIO集群可以组成联邦,并形成一个全局的命名空间,并跨越多个数据中心;
  • SDK支持: 基于Minio轻量的特点,它得到类似Java、Python或Go等语言的sdk支持;
  • 支持纠删码:MinIO使用纠删码、Checksum来防止硬件错误和静默数据污染。在最高冗余度配置下,即使丢失1/2的磁盘也能恢复数据;

与FastDFS对比

  • FastDFS是阿里余庆做的个人项目,也是一款开源高性能的分布式文件系统,适合小规模文件数据存储,默认不提供UI界面,安装部署(运维)复杂,很难达到以G为单位的每秒读写速度,没有完备的官方文档,环境搭建较为复杂;
  • MinIO是由MinIO.Inc运营的开源项目,号称世界上速度最快的对象存储服务器,并且社区活跃度高,标准硬件条件下它能达到55GB/s的读、35GB/s的写速率,而且MinIO部署自带管理界面,不需要额外安装;MinIO提供了所有主流开发语言的SDK,并且兼容亚马逊S3云存储服务接口,在MinIO中一个对象文件可以是任意大小,从几KB到最大的5T不等;最后它提供了与k8s、etcd、docker等容器技术深度集成方案,可以说就是为云原生而生的。(缺点,不支持动态增加节点)

MinIO快速使用

1、下载MinIO安装包

下载地址:https://www.minio.org.cn/download.shtml#/windows

备用地址:https://dl.minio.io/server/minio/release/windows-amd64/minio.exe

2、创建数据存储文件夹data

3、在minio.exe文件夹的路径处输入cmd进入命令行界面,启动MinIO服务:

minio.exe server .\data

命令行输出:

Formatting 1st pool, 1 set(s), 1 drives per set.
WARNING: Host local has more than 0 drives of set. A host failure will result in data becoming unavailable.
WARNING: Detected default credentials 'minioadmin:minioadmin', we recommend that you change these values with 'MINIO_ROOT_USER' and 'MINIO_ROOT_PASSWORD' environment variables
MinIO Object Storage Server
Copyright: 2015-2023 MinIO, Inc.
License: GNU AGPLv3 <https://www.gnu.org/licenses/agpl-3.0.html>
Version: RELEASE.2023-08-04T17-40-21Z (go1.19.12 windows/amd64)

Status:         1 Online, 0 Offline.
S3-API: http://192.168.10.41:9000  http://192.168.140.1:9000  http://192.168.46.1:9000  http://127.0.0.1:9000
RootUser: minioadmin
RootPass: minioadmin

Console: http://192.168.10.41:53081 http://192.168.140.1:53081 http://192.168.46.1:53081 http://127.0.0.1:53081
RootUser: minioadmin
RootPass: minioadmin

Command-line: https://min.io/docs/minio/linux/reference/minio-mc.html#quickstart
   $ mc.exe alias set myminio http://192.168.10.41:9000 minioadmin minioadmin

Documentation: https://min.io/docs/minio/linux/index.html
Warning: The standard parity is set to 0. This can lead to data loss.
IAM refresh took 19.79s

4、浏览器访问http://127.0.0.1:53081,输入账号minioadmin,密码minioadmin即可登录。

5、创建桶,可将桶看作文件夹,这里我创建了一个fangfile桶,设置Access Policy为public。

6、创建Access Key,进入Access Keys页面,点Create access key,记录Access Key和Secret Key。

Spring Boot集成MinIO

1、创建Spring Boot应用

2、pom文件添加相关依赖

<parent>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-parent</artifactId>
    <version>2.2.1.RELEASE</version>
    <relativePath/> <!-- lookup parent from repository -->
</parent>

<groupId>com.fang</groupId>
<artifactId>minio-demo</artifactId>
<version>0.0.1-SNAPSHOT</version>
<name>minio-demo</name>
<description>Demo project for Spring Boot</description>
<properties>
    <java.version>1.8</java.version>
</properties>
<dependencies>
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-web</artifactId>
    </dependency>

    <dependency>
        <groupId>org.projectlombok</groupId>
        <artifactId>lombok</artifactId>
        <optional>true</optional>
    </dependency>
    <dependency>
        <groupId>io.minio</groupId>
        <artifactId>minio</artifactId>
        <version>7.1.0</version>
    </dependency>
    <dependency>
        <groupId>com.squareup.okhttp3</groupId>
        <artifactId>okhttp</artifactId>
        <version>3.14.9</version>
    </dependency>
    <dependency>
        <groupId>com.github.xiaoymin</groupId>
        <artifactId>knife4j-spring-boot-starter</artifactId>
        <version>2.0.5</version>
    </dependency>
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-test</artifactId>
        <scope>test</scope>
    </dependency>
</dependencies>

3、整合Swagger,编写Swagger配置类

package com.fang.miniodemo.config;

import com.github.xiaoymin.knife4j.spring.annotations.EnableKnife4j;
import org.springframework.context.annotation.Configuration;
import springfox.documentation.builders.ApiInfoBuilder;
import springfox.documentation.builders.PathSelectors;
import springfox.documentation.builders.RequestHandlerSelectors;
import springfox.documentation.service.ApiInfo;
import springfox.documentation.service.Contact;
import springfox.documentation.spi.DocumentationType;
import springfox.documentation.spring.web.plugins.Docket;
import springfox.documentation.swagger2.annotations.EnableSwagger2;

@Configuration
@EnableSwagger2
@EnableKnife4j
public class SwaggerConfig {
    public Docket webApiConfig() {
        return new Docket(DocumentationType.SWAGGER_2)
                .groupName("webApi")
                .apiInfo(webApiInfo())
                .select()
                .apis(RequestHandlerSelectors.basePackage("com.fang.controller"))
                .paths(PathSelectors.any())
                .build();
    }

    private ApiInfo webApiInfo() {
        return new ApiInfoBuilder()
                .title("网站-API文档")
                .description("微服务接口定义")
                .version("1.0")
                .contact(new Contact("方伟俊", "", "568036792@qq.com"))
                .build();
    }
}

4、编写MinIO属性配置类

package com.fang.miniodemo.config;

import lombok.Data;
import org.springframework.boot.context.properties.ConfigurationProperties;
import org.springframework.stereotype.Component;

import java.io.Serializable;

@Data
@Component
@ConfigurationProperties(prefix = "minio")
public class MinIOConfigProperties implements Serializable {
    private String accessKey;
    private String secretKey;
    private String bucket;
    private String endpoint;
    private String readPath;
}

5、编写MinIO配置类,注册MinioClient客户端Bean对象

package com.fang.miniodemo.config;

import io.minio.MinioClient;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

@Configuration
public class MinIOConfig {
    @Autowired
    private MinIOConfigProperties minIOConfigProperties;

    @Bean
    public MinioClient buildMinioClient() {
        return MinioClient.builder()
                .credentials(minIOConfigProperties.getAccessKey(), minIOConfigProperties.getSecretKey())
                .endpoint(minIOConfigProperties.getEndpoint())
                .build();
    }
}

6、在application.properties文件配置minio自定义属性

minio.accessKey=XM5jigwjjwMMWAb9tHEf
minio.secretKey=rZyoUixUUtVDDl8O9TSvszFXl1ODbeYL9t3gJOMe
minio.bucket=fangfile
minio.endpoint=http://192.168.10.41:9000
minio.readPath=http://192.168.10.41:9000

7、编写minio业务接口

package com.fang.miniodemo.service;

import java.io.InputStream;

public interface FileStorageService {
    String uploadImgFile(String prefix, String filename, InputStream inputStream);
}

业务接口实现类:

package com.fang.miniodemo.service.impl;

import com.fang.miniodemo.config.MinIOConfigProperties;
import com.fang.miniodemo.service.FileStorageService;
import io.minio.MinioClient;
import io.minio.PutObjectArgs;
import lombok.extern.slf4j.Slf4j;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Service;
import org.springframework.util.StringUtils;

import java.io.InputStream;
import java.text.SimpleDateFormat;
import java.util.Date;

@Slf4j
@Service
public class MinIOFileStorageService implements FileStorageService {
    @Autowired
    private MinioClient minioClient;

    @Autowired
    private MinIOConfigProperties minIOConfigProperties;
    private final static String separator = "/"; //文件夹分隔符
    /**
     * 构建文件的绝对路径
     *
     * @param dirPath  文件路径
     * @param filename 文件名  yyyy/mm/dd/file.jpg
     * @return /test
     */
    public String builderFilePath(String dirPath, String filename) {
        StringBuilder stringBuilder = new StringBuilder(50);
        if (!StringUtils.isEmpty(dirPath)) {
            stringBuilder.append(dirPath).append(separator);
        }
        SimpleDateFormat sdf = new SimpleDateFormat("yyyy/MM/dd");
        String todayStr = sdf.format(new Date());
        stringBuilder.append(todayStr).append(separator);
        stringBuilder.append(filename);
        return stringBuilder.toString();
    }
    @Override
    public String uploadImgFile(String prefix, String filename, InputStream inputStream) {
        String filePath = builderFilePath(prefix, filename);
        try {

            PutObjectArgs putObjectArgs = PutObjectArgs.builder()
                    .object(filePath)
                    .contentType("image/jpg")
                    .bucket(minIOConfigProperties.getBucket())
                    .stream(inputStream, inputStream.available(), -1)
                    .build();
            minioClient.putObject(putObjectArgs);
            StringBuilder urlPath = new StringBuilder(minIOConfigProperties.getReadPath());
            urlPath.append(separator + minIOConfigProperties.getBucket());
            urlPath.append(separator);
            urlPath.append(filePath);
            return urlPath.toString();
        } catch (Exception ex) {
            log.error("minio put file error: ", ex);
            throw new RuntimeException("上传文件失败");
        }
    }
}

8、编写统一结果处理类

package com.fang.miniodemo.dto;

import lombok.Data;

@Data
public class Result<T> {
    private Integer code; //响应状态码
    private String msg;   //响应消息
    private T data;       //响应数据

    /**
     * 处理成功的返回结果
     *
     * @param data 数据
     * @param <T>
     * @return R<T>
     */
    public static <T> Result<T> success(T data, String msg) {
        Result<T> r = new Result<>();
        r.setCode(200);
        r.setData(data);
        r.setMsg(msg);
        return r;
    }

    /**
     * 处理成功的返回结果
     *
     * @param msg 错误信息
     * @param <T>
     * @return R<T>
     */
    public static <T> Result<T> error(String msg) {
        Result<T> r = new Result<>();
        r.setCode(503);
        r.setMsg(msg);
        return r;
    }
}

9、编写Controller类

package com.fang.miniodemo.controller;

import com.fang.miniodemo.dto.Result;
import com.fang.miniodemo.service.FileStorageService;
import io.swagger.annotations.Api;
import io.swagger.annotations.ApiOperation;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;
import org.springframework.web.multipart.MultipartFile;

import java.io.IOException;
import java.io.InputStream;
import java.util.UUID;

@RestController
@RequestMapping("/minio")
@Api(tags = "minio相关接口")
public class MinioController {
    @Autowired
    private FileStorageService fileStorageService;


    /**
     * 上传图片到minio
     *
     * @param file
     * @return
     */
    @PostMapping("upload")
    @ApiOperation(value = "图片上传接口")
    public Result uploadFile(MultipartFile file) throws IOException {
        try {
            // 获取文件名称
            String fileName = file.getOriginalFilename();
            /*解决多次上传同名文件覆盖问题*/
            // 在文件名称里面添加随机唯一的值
            String uuid = UUID.randomUUID().toString().replaceAll("-", "");
            fileName = uuid + fileName;
            // 获取文件输入流
            InputStream is = file.getInputStream();
            String imgUrl = fileStorageService.uploadImgFile("img", fileName, is);
            return Result.success(imgUrl, "上传成功");
        } catch (IOException e) {
            e.printStackTrace();
            return Result.error("上传失败");
        }
    }
}

10、启动项目,通过Swagger调试界面上传文件测试,上传后可以看到返回的文件路径,可以看到MinIO界面已上传的文件。

Q&A

1、官网教程Maven引用MinIO包后,启动时报错:程序包io.minio不存在

<dependency>
    <groupId>io.minio</groupId>
    <artifactId>minio</artifactId>
    <version>8.5.4</version>
</dependency>

解决:首先到maven仓库找到对应的jar包并下载到本地,maven官网

这里下载的是7.1.0版本,在命令行执行以下命令:

mvn install:install-file -Dfile=minio-7.1.0.jar -DgroupId=io.minio -DartifactId=minio -Dversion=7.1.0 -Dpackaging=jar
-Dfile:从仓库下载的jar包的存放路径
-DgroupId:对应依赖的groupId
-DartifactId:对应依赖的artifactId
-Dversion:对应依赖的version
-Dpackaging:对应文件类型 jar

安装成功后,将Spring Boot工程里minio的依赖改成7.1.0。

2、解决完以上问题后,启动时报错:MinIOConfig.java: 无法访问okhttp3.HttpUrl,找不到okhttp3.HttpUrl的类文件

解决:添加以下依赖:

<dependency>
    <groupId>com.squareup.okhttp3</groupId>
    <artifactId>okhttp</artifactId>
    <version>3.14.9</version>
</dependency>

3、启动时okhttp3报error in opening zip file错误

解决:删除maven本地仓库的jar包,然后idea中项目刷新,让maven重新下载依赖。

参考资料