Fading Coder

One Final Commit for the Last Sprint

Home > Tools > Content

Dynamic Schema Management and Runtime Updates in Apache ShardingSphere

Tools May 2 13

A data node represents the atomic unit of sharded storage, defined by a specific source connection paired with a physical table name. For example: ds_inventory.products_0.

Static Configuraton Methods

1. Manual Enumeration

The simplest approach involves listing every target node explicitly within the configuration file.

sharding:
  tables:
    t_orders:
      actual-data-nodes: prod_ds.order_a, prod_ds.order_b, backup_ds.order_c

2. Groovy Inline Syntax

ShardingSphere supports expressions based on Groovy to generate sequences programmatically.

actual-data-nodes: source_db.log_${0..5}

This syntax automatically expands into six distinct nodes (0 through 5). Advanced range logic is possible but requires understanding Groovy evaluation rules.

3. External Method Calls

For custom logic that cannot be expressed inline, you may invoke static utility methods directly from the YAML definition.

actual-data-nodes: ds_main.data_$->{com.app.utils.NodeFactory.createNodes()}

Programmatic Rule Definition

Hardcoding metadata restricts adaptability. Using the Java API allows retrieving active tables from a database or cache to construct the routing rules dynamically.

public String deriveNodeList(String dbPrefix, String logicalTable) {
    List<String> physicalTables = dao.queryActiveTables(logicalTable);
    StringBuilder sb = new StringBuilder();
    for (int i = 0; i < physicalTables.size(); i++) {
        sb.append(dbPrefix).append(".").append(physicalTables.get(i));
        if (i < physicalTables.size() - 1) sb.append(",");
    }
    return sb.toString();
}

This approach enables the application to reflect real-world schema changes without manual code deployment.

Hot Reload Mechenism

Standard YAML updates require restarting the application to reload configuration. To update sharding rules while the service remains running, the internal state management must be triggered manually.

Legacy Context Retrieval

In versions prior to 5.x, the context manager was accessible via public APIs.

ContextManager mgr = dataSource.getContextManager();
mgr.alterRuleConfiguration(schemaName, newRules);

Modern Encapsulation (5.x+)

Newer versions encapsulate the contextManager. Accessing it requires using Java Reflection to bypass access restrictions safely.

private ContextManager getInternalContext(ShardingSphereDataSource ds) throws ReflectiveOperationException {
    Field field = ShardingSphereDataSource.class.getDeclaredField("contextManager");
    field.setAccessible(true);
    return (ContextManager) field.get(ds);
}

Complete Implementation Example

The following class demonstrates how to combine dynamic node building with hot-reloading. It fetches configurations, constructs new rules, and applies them instantly via relfection.

@Service
public class ShardingTopologyReloader {

    @Autowired
    private ShardingSphereDataSource dataSource;

    public void refreshTopology(List<TableDefinitionVO> definitions) throws Exception {
        // Retrieve kernel state via reflection
        ContextManager cm = getInternalContext(dataSource);
        
        List<RuleConfiguration> updatedConfigs = new ArrayList<>();
        Map<String, AlgorithmConfiguration> algoMap = new HashMap<>();
        
        for (TableDefinitionVO def : definitions) {
            String nodesStr = buildNodes(def.getSourceDb(), def.getLogicTable());
            
            ShardingTableRuleConfiguration ruleConfig = new ShardingTableRuleConfiguration(
                def.getLogicTable(), 
                nodesStr
            );
            
            String algName = def.getLogicTable() + "_alg";
            ruleConfig.setTableShardingStrategy(new StandardShardingStrategyConfiguration(
                def.getSplitColumn(), 
                algName
            ));
            
            Properties props = new Properties();
            props.put("lowerBound", def.getDateLower());
            algoMap.put(algName, new AlgorithmConfiguration("INLINE", props));
            
            updatedConfigs.add(ruleConfig);
        }
        
        // Build master configuration object
        ShardingRuleConfiguration fullConfig = new ShardingRuleConfiguration();
        fullConfig.setTables(updatedConfigs);
        fullConfig.setShardingAlgorithms(algoMap);
        
        long start = System.currentTimeMillis();
        cm.alterRuleConfiguration("business_schema", Arrays.asList(fullConfig));
        log.info("Routing topology refreshed in {}ms", System.currentTimeMillis() - start);
    }
    
    private String buildNodes(String dbName, String tableName) {
        List<String> tables = repository.findPhysicalNames(tableName);
        return tables.stream()
            .map(t -> dbName + "." + t)
            .collect(Collectors.joining(","));
    }
}

Related Articles

Efficient Usage of HTTP Client in IntelliJ IDEA

IntelliJ IDEA incorporates a versatile HTTP client tool, enabling developres to interact with RESTful services and APIs effectively with in the editor. This functionality streamlines workflows, replac...

Resolve PhpStorm "Interpreter is not specified or invalid" on WAMP (Windows)

Symptom PhpStorm displays: "Interpreter is not specified or invalid. Press ‘Fix’ to edit your project configuration." This occurs when the IDE cannot locate a valid PHP CLI executable or when the debu...

Capturing Android Screenshots and Screen Recordings with ADB

Two practical ways to grab images and videos from an Android device: Mirror the phone display to a computer and use desktop tools for screenshots and GIFs Use ADB commands (no UI mirroring required)...

Leave a Comment

Anonymous

◎Feel free to join the discussion and share your thoughts.