Spring Boot 集成百度 Uidgenerator分布式ID生成器

UidGenerator是Java实现的, 基于Snowflake算法的唯一ID生成器。UidGenerator以组件形式工作在应用项目中, 支持自定义workerId位数和初始化策略, 从而适用于docker等虚拟化环境下实例自动重启、漂移等场景。在实现上, UidGenerator通过借用未来时间来解决sequence天然存在的并发限制; 采用RingBuffer来缓存已生成的UID, 并行化UID的生产和消费, 同时对CacheLine补齐，避免了由RingBuffer带来的硬件级「伪共享」问题. 最终单机QPS可达600万。

官方地址：https://github.com/baidu/uid-generator

uid-generator提供了两种生成器: DefaultUidGenerator、CachedUidGenerator。如对UID生成性能有要求, 请使用CachedUidGenerator

CachedUidGenerator

RingBuffer环形数组，数组每个元素成为一个slot。RingBuffer容量，默认为Snowflake算法中sequence最大值，且为2^N。可通过boostPower配置进行扩容，以提高RingBuffer 读写吞吐量。

Tail指针、Cursor指针用于环形数组上读写slot：

Tail指针
表示Producer生产的最大序号(此序号从0开始，持续递增)。Tail不能超过Cursor，即生产者不能覆盖未消费的slot。当Tail已赶上curosr，此时可通过rejectedPutBufferHandler指定PutRejectPolicy
Cursor指针
表示Consumer消费到的最小序号(序号序列与Producer序列相同)。Cursor不能超过Tail，即不能消费未生产的slot。当Cursor已赶上tail，此时可通过rejectedTakeBufferHandler指定TakeRejectPolicy

CachedUidGenerator采用了双RingBuffer，Uid-RingBuffer用于存储Uid、Flag-RingBuffer用于存储Uid状态(是否可填充、是否可消费)

由于数组元素在内存中是连续分配的，可最大程度利用CPU cache以提升性能。但同时会带来「伪共享」FalseSharing问题，为此在Tail、Cursor指针、Flag-RingBuffer中采用了CacheLine 补齐方式。

RingBuffer填充时机

初始化预填充
RingBuffer初始化时，预先填充满整个RingBuffer.
即时填充
Take消费时，即时检查剩余可用slot量(tail - cursor)，如小于设定阈值，则补全空闲slots。阈值可通过paddingFactor来进行配置，请参考Quick Start中CachedUidGenerator配置
周期填充
通过Schedule线程，定时补全空闲slots。可通过scheduleInterval配置，以应用定时填充功能，并指定Schedule时间间隔

1、导入依赖

<!-- https://mvnrepository.com/artifact/com.xfvape.uid/uid-generator -->
<dependency><groupId>com.xfvape.uid</groupId><artifactId>uid-generator</artifactId><version>0.0.4-RELEASE</version>
</dependency>

2、创建表WORKER_NODE

运行sql脚本以导入表WORKER_NODE, 脚本如下:

DROP DATABASE IF EXISTS `xxxx`;
CREATE DATABASE `xxxx` ;
use `xxxx`;
DROP TABLE IF EXISTS WORKER_NODE;
CREATE TABLE WORKER_NODE
(
ID BIGINT NOT NULL AUTO_INCREMENT COMMENT 'auto increment id',
HOST_NAME VARCHAR(64) NOT NULL COMMENT 'host name',
PORT VARCHAR(64) NOT NULL COMMENT 'port',
TYPE INT NOT NULL COMMENT 'node type: ACTUAL or CONTAINER',
LAUNCH_DATE DATE NOT NULL COMMENT 'launch date',
MODIFIED TIMESTAMP NOT NULL COMMENT 'modified time',
CREATED TIMESTAMP NOT NULL COMMENT 'created time',
PRIMARY KEY(ID)
)COMMENT='DB WorkerID Assigner for UID Generator',ENGINE = INNODB;

3、maven依赖

 <dependencies><dependency><groupId>com.xiaobear.distributedid</groupId><artifactId>distributedid-core</artifactId><version>1.0-SNAPSHOT</version></dependency><dependency><groupId>org.springframework.boot</groupId><artifactId>spring-boot-starter-web</artifactId></dependency><dependency><groupId>org.springframework.boot</groupId><artifactId>spring-boot-starter-jdbc</artifactId></dependency><dependency><groupId>mysql</groupId><artifactId>mysql-connector-java</artifactId><scope>runtime</scope></dependency><dependency><groupId>org.mybatis.spring.boot</groupId><artifactId>mybatis-spring-boot-starter</artifactId><version>2.1.0</version></dependency><!--必须放在最后--><!-- https://mvnrepository.com/artifact/com.xfvape.uid/uid-generator --><dependency><groupId>com.xfvape.uid</groupId><artifactId>uid-generator</artifactId><version>0.0.4-RELEASE</version><exclusions><exclusion><groupId>org.slf4j</groupId><artifactId>log4j-over-slf4j</artifactId></exclusion><exclusion><groupId>ch.qos.logback</groupId><artifactId>logback-classic</artifactId></exclusion><exclusion><groupId>org.slf4j</groupId><artifactId>slf4j-api</artifactId></exclusion><exclusion><groupId>org.mybatis</groupId><artifactId>mybatis-spring</artifactId></exclusion><exclusion><groupId>org.mybatis</groupId><artifactId>mybatis</artifactId></exclusion></exclusions></dependency></dependencies>

4、配置文件

spring:datasource:type: com.zaxxer.hikari.HikariDataSourceurl: jdbc:mysql://127.0.0.1:3306/distributedid?serverTimezone=Asia/Shanghai&characterEncoding=utf8&useUnicode=true&useSSL=false&autoReconnect=trueusername: rootpassword: 密码driver-class-name: com.mysql.cj.jdbc.Driver
server:port: 8088mybatis:type-aliases-package: com.xiaobear.distributedid.core.domainmapper-locations: classpath*:mapper/*.xml

5、主启动类

@SpringBootApplication
@MapperScan("com.xiaobear.distributedid.mapper")
public class UidgeneratorApplication {public static void main(String[] args) {SpringApplication.run(UidgeneratorApplication.class);System.out.println("百度生成ID 启动成功");}
}

6、业务类

1、实体类

public class WorkerNode {private Long id;private String hostName;private String port;private Integer type;private Date launchDate;private Date modified;private Date created;public Long getId() {return id;}public void setId(Long id) {this.id = id;}public String getHostName() {return hostName;}public void setHostName(String hostName) {this.hostName = hostName;}public String getPort() {return port;}public void setPort(String port) {this.port = port;}public Integer getType() {return type;}public void setType(Integer type) {this.type = type;}public Date getLaunchDate() {return launchDate;}public void setLaunchDate(Date launchDate) {this.launchDate = launchDate;}public Date getModified() {return modified;}public void setModified(Date modified) {this.modified = modified;}public Date getCreated() {return created;}public void setCreated(Date created) {this.created = created;}@Overridepublic String toString() {return "WorkerNode{" +"id=" + id +", hostName='" + hostName + '\'' +", port='" + port + '\'' +", type=" + type +", launchDate=" + launchDate +", modified=" + modified +", created=" + created +'}';}
}

2、数据层接口

@Mapper
public interface WorkerNodeMapper {/*** 添加对象* @param workerNodeEntity* @return*/int addWorkerNode(WorkerNode workerNodeEntity);/*** 通过host port 获取ID* @param host* @param port* @return*/WorkerNode getWorkerNodeByHostPort(@Param("host") String host, @Param("port") String port);
}

mapper.xml

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE mapper PUBLIC "-//mybatis.org//DTD Mapper 3.0//EN" "http://mybatis.org/dtd/mybatis-3-mapper.dtd">
<mapper namespace="com.xiaobear.distributedid.mapper.WorkerNodeMapper"><resultMap id="workerNodeRes" type="com.xiaobear.distributedid.core.domain.WorkerNode"><id column="ID" jdbcType="BIGINT" property="id"/><result column="HOST_NAME" jdbcType="VARCHAR" property="hostName"/><result column="PORT" jdbcType="VARCHAR" property="port"/><result column="TYPE" jdbcType="INTEGER" property="type"/><result column="LAUNCH_DATE" jdbcType="DATE" property="launchDate"/><result column="MODIFIED" jdbcType="TIMESTAMP" property="modified"/><result column="CREATED" jdbcType="TIMESTAMP" property="created"/></resultMap><insert id="addWorkerNode" useGeneratedKeys="true" keyProperty="id"parameterType="com.xiaobear.distributedid.core.domain.WorkerNode">INSERT INTO WORKER_NODE(HOST_NAME,PORT,TYPE,LAUNCH_DATE,MODIFIED,CREATED)VALUES (#{hostName},#{port},#{type},#{launchDate},NOW(),NOW())</insert><select id="getWorkerNodeByHostPort" resultMap="workerNodeRes">SELECTID,HOST_NAME,PORT,TYPE,LAUNCH_DATE,MODIFIED,CREATEDFROMWORKER_NODEWHEREHOST_NAME = #{host} AND PORT = #{port}</select>
</mapper>

3、实现层接口

public interface IWorkerNodeService {/*** 获取ID* @return*/public long genUid();
}

4、实现层实现类

@Service
public class WorkerNodeServiceImpl implements IWorkerNodeService {@Resourceprivate UidGenerator uidGenerator;/*** 获取百度生成ID* @return*/@Overridepublic long genUid() {return uidGenerator.getUID();}
}

5、控制层接口

@RestController
public class WorkerNodeController {@Resourceprivate IWorkerNodeService workerNodeService;/***使用百度 Uidgenerator获取ID* @return*/@GetMapping("/Uidgenerator")public RestResponse getIdByBaiDuUid(){long id = workerNodeService.genUid();return RestResponse.success(id);}
}

7、测试

http://localhost:8088/Uidgenerator

{"code":200,"msg":"成功","result":7067247636209745920}

8、uid-generator核心对象装配为spring的bean

1、重写WorkerIdAssigner接口

public class DisposableWorkerIdAssigner implements WorkerIdAssigner{@Resourceprivate WorkerNodeMapper workerNodeMapper;@Override@Transactional(rollbackFor = Exception.class)public long assignWorkerId() {WorkerNode workerNode = buildWorkerNode();workerNodeMapper.addWorkerNode(workerNode);return workerNode.getId();}private WorkerNode buildWorkerNode() {WorkerNode workNode = new WorkerNode();if (DockerUtils.isDocker()) {workNode.setType(WorkerNodeType.CONTAINER.value());workNode.setHostName(DockerUtils.getDockerHost());workNode.setPort(DockerUtils.getDockerPort());} else {workNode.setType(WorkerNodeType.ACTUAL.value());workNode.setHostName(NetUtils.getLocalAddress());workNode.setPort(System.currentTimeMillis() + "-" + RandomUtils.nextInt(100000));}workNode.setLaunchDate(new Date());return workNode;}
}

2、自动转配bean

@Configuration
public class WorkerNodeConfig {@Bean(name = "disposableWorkerIdAssigner")public DisposableWorkerIdAssigner disposableWorkerIdAssigner(){return new DisposableWorkerIdAssigner();}@Bean(name = "cachedUidGenerator")public UidGenerator uidGenerator(DisposableWorkerIdAssigner disposableWorkerIdAssigner){CachedUidGenerator cachedUidGenerator = new CachedUidGenerator();cachedUidGenerator.setWorkerIdAssigner(disposableWorkerIdAssigner);return cachedUidGenerator;}
}

3、使用，在实现类指定生成ID的bean

@Service
public class WorkerNodeServiceImpl implements IWorkerNodeService {@Resource(name = "cachedUidGenerator")private UidGenerator uidGenerator;/*** 获取百度生成ID* @return*/@Overridepublic long genUid() {return uidGenerator.getUID();}
}