Fading Coder

One Final Commit for the Last Sprint

Home > Tools > Content

Dynamic Schema Management and Runtime Updates in Apache ShardingSphere

Tools 2

A data node represents the atomic unit of sharded storage, defined by a specific source connection paired with a physical table name. For example: ds_inventory.products_0.

Static Configuraton Methods

1. Manual Enumeration

The simplest approach involves listing every target node explicitly within the configuration file.

sharding:
  tables:
    t_orders:
      actual-data-nodes: prod_ds.order_a, prod_ds.order_b, backup_ds.order_c

2. Groovy Inline Syntax

ShardingSphere supports expressions based on Groovy to generate sequences programmatically.

actual-data-nodes: source_db.log_${0..5}

This syntax automatically expands into six distinct nodes (0 through 5). Advanced range logic is possible but requires understanding Groovy evaluation rules.

3. External Method Calls

For custom logic that cannot be expressed inline, you may invoke static utility methods directly from the YAML definition.

actual-data-nodes: ds_main.data_$->{com.app.utils.NodeFactory.createNodes()}

Programmatic Rule Definition

Hardcoding metadata restricts adaptability. Using the Java API allows retrieving active tables from a database or cache to construct the routing rules dynamically.

public String deriveNodeList(String dbPrefix, String logicalTable) {
    List<String> physicalTables = dao.queryActiveTables(logicalTable);
    StringBuilder sb = new StringBuilder();
    for (int i = 0; i < physicalTables.size(); i++) {
        sb.append(dbPrefix).append(".").append(physicalTables.get(i));
        if (i < physicalTables.size() - 1) sb.append(",");
    }
    return sb.toString();
}

This approach enables the application to reflect real-world schema changes without manual code deployment.

Hot Reload Mechenism

Standard YAML updates require restarting the application to reload configuration. To update sharding rules while the service remains running, the internal state management must be triggered manually.

Legacy Context Retrieval

In versions prior to 5.x, the context manager was accessible via public APIs.

ContextManager mgr = dataSource.getContextManager();
mgr.alterRuleConfiguration(schemaName, newRules);

Modern Encapsulation (5.x+)

Newer versions encapsulate the contextManager. Accessing it requires using Java Reflection to bypass access restrictions safely.

private ContextManager getInternalContext(ShardingSphereDataSource ds) throws ReflectiveOperationException {
    Field field = ShardingSphereDataSource.class.getDeclaredField("contextManager");
    field.setAccessible(true);
    return (ContextManager) field.get(ds);
}

Complete Implementation Example

The following class demonstrates how to combine dynamic node building with hot-reloading. It fetches configurations, constructs new rules, and applies them instantly via relfection.

@Service
public class ShardingTopologyReloader {

    @Autowired
    private ShardingSphereDataSource dataSource;

    public void refreshTopology(List<TableDefinitionVO> definitions) throws Exception {
        // Retrieve kernel state via reflection
        ContextManager cm = getInternalContext(dataSource);
        
        List<RuleConfiguration> updatedConfigs = new ArrayList<>();
        Map<String, AlgorithmConfiguration> algoMap = new HashMap<>();
        
        for (TableDefinitionVO def : definitions) {
            String nodesStr = buildNodes(def.getSourceDb(), def.getLogicTable());
            
            ShardingTableRuleConfiguration ruleConfig = new ShardingTableRuleConfiguration(
                def.getLogicTable(), 
                nodesStr
            );
            
            String algName = def.getLogicTable() + "_alg";
            ruleConfig.setTableShardingStrategy(new StandardShardingStrategyConfiguration(
                def.getSplitColumn(), 
                algName
            ));
            
            Properties props = new Properties();
            props.put("lowerBound", def.getDateLower());
            algoMap.put(algName, new AlgorithmConfiguration("INLINE", props));
            
            updatedConfigs.add(ruleConfig);
        }
        
        // Build master configuration object
        ShardingRuleConfiguration fullConfig = new ShardingRuleConfiguration();
        fullConfig.setTables(updatedConfigs);
        fullConfig.setShardingAlgorithms(algoMap);
        
        long start = System.currentTimeMillis();
        cm.alterRuleConfiguration("business_schema", Arrays.asList(fullConfig));
        log.info("Routing topology refreshed in {}ms", System.currentTimeMillis() - start);
    }
    
    private String buildNodes(String dbName, String tableName) {
        List<String> tables = repository.findPhysicalNames(tableName);
        return tables.stream()
            .map(t -> dbName + "." + t)
            .collect(Collectors.joining(","));
    }
}

Related Articles

Efficient Usage of HTTP Client in IntelliJ IDEA

IntelliJ IDEA incorporates a versatile HTTP client tool, enabling developres to interact with RESTful services and APIs effectively with in the editor. This functionality streamlines workflows, replac...

Installing CocoaPods on macOS Catalina (10.15) Using a User-Managed Ruby

System Ruby on macOS 10.15 frequently fails to build native gems required by CocoaPods (for example, ffi), leading to errors like: ERROR: Failed to build gem native extension checking for ffi.h... no...

Resolve PhpStorm "Interpreter is not specified or invalid" on WAMP (Windows)

Symptom PhpStorm displays: "Interpreter is not specified or invalid. Press ‘Fix’ to edit your project configuration." This occurs when the IDE cannot locate a valid PHP CLI executable or when the debu...

Leave a Comment

Anonymous

◎Feel free to join the discussion and share your thoughts.