Dynamic Schema Management and Runtime Updates in Apache ShardingSphere
A data node represents the atomic unit of sharded storage, defined by a specific source connection paired with a physical table name. For example: ds_inventory.products_0.
Static Configuraton Methods
1. Manual Enumeration
The simplest approach involves listing every target node explicitly within the configuration file.
sharding:
tables:
t_orders:
actual-data-nodes: prod_ds.order_a, prod_ds.order_b, backup_ds.order_c
2. Groovy Inline Syntax
ShardingSphere supports expressions based on Groovy to generate sequences programmatically.
actual-data-nodes: source_db.log_${0..5}
This syntax automatically expands into six distinct nodes (0 through 5). Advanced range logic is possible but requires understanding Groovy evaluation rules.
3. External Method Calls
For custom logic that cannot be expressed inline, you may invoke static utility methods directly from the YAML definition.
actual-data-nodes: ds_main.data_$->{com.app.utils.NodeFactory.createNodes()}
Programmatic Rule Definition
Hardcoding metadata restricts adaptability. Using the Java API allows retrieving active tables from a database or cache to construct the routing rules dynamically.
public String deriveNodeList(String dbPrefix, String logicalTable) {
List<String> physicalTables = dao.queryActiveTables(logicalTable);
StringBuilder sb = new StringBuilder();
for (int i = 0; i < physicalTables.size(); i++) {
sb.append(dbPrefix).append(".").append(physicalTables.get(i));
if (i < physicalTables.size() - 1) sb.append(",");
}
return sb.toString();
}
This approach enables the application to reflect real-world schema changes without manual code deployment.
Hot Reload Mechenism
Standard YAML updates require restarting the application to reload configuration. To update sharding rules while the service remains running, the internal state management must be triggered manually.
Legacy Context Retrieval
In versions prior to 5.x, the context manager was accessible via public APIs.
ContextManager mgr = dataSource.getContextManager();
mgr.alterRuleConfiguration(schemaName, newRules);
Modern Encapsulation (5.x+)
Newer versions encapsulate the contextManager. Accessing it requires using Java Reflection to bypass access restrictions safely.
private ContextManager getInternalContext(ShardingSphereDataSource ds) throws ReflectiveOperationException {
Field field = ShardingSphereDataSource.class.getDeclaredField("contextManager");
field.setAccessible(true);
return (ContextManager) field.get(ds);
}
Complete Implementation Example
The following class demonstrates how to combine dynamic node building with hot-reloading. It fetches configurations, constructs new rules, and applies them instantly via relfection.
@Service
public class ShardingTopologyReloader {
@Autowired
private ShardingSphereDataSource dataSource;
public void refreshTopology(List<TableDefinitionVO> definitions) throws Exception {
// Retrieve kernel state via reflection
ContextManager cm = getInternalContext(dataSource);
List<RuleConfiguration> updatedConfigs = new ArrayList<>();
Map<String, AlgorithmConfiguration> algoMap = new HashMap<>();
for (TableDefinitionVO def : definitions) {
String nodesStr = buildNodes(def.getSourceDb(), def.getLogicTable());
ShardingTableRuleConfiguration ruleConfig = new ShardingTableRuleConfiguration(
def.getLogicTable(),
nodesStr
);
String algName = def.getLogicTable() + "_alg";
ruleConfig.setTableShardingStrategy(new StandardShardingStrategyConfiguration(
def.getSplitColumn(),
algName
));
Properties props = new Properties();
props.put("lowerBound", def.getDateLower());
algoMap.put(algName, new AlgorithmConfiguration("INLINE", props));
updatedConfigs.add(ruleConfig);
}
// Build master configuration object
ShardingRuleConfiguration fullConfig = new ShardingRuleConfiguration();
fullConfig.setTables(updatedConfigs);
fullConfig.setShardingAlgorithms(algoMap);
long start = System.currentTimeMillis();
cm.alterRuleConfiguration("business_schema", Arrays.asList(fullConfig));
log.info("Routing topology refreshed in {}ms", System.currentTimeMillis() - start);
}
private String buildNodes(String dbName, String tableName) {
List<String> tables = repository.findPhysicalNames(tableName);
return tables.stream()
.map(t -> dbName + "." + t)
.collect(Collectors.joining(","));
}
}