Fading Coder

One Final Commit for the Last Sprint

Home > Tech > Content

MyCat Database Sharding Strategies

Tech May 19 1

Configuration Rules

MyCat sharding requires defining table rules and functions in configuration files.

Table Rule Structure

<tableRule name="custom-sharding-long">
    <rule>
        <columns>user_id</columns>
        <algorithm>range-long</algorithm>
    </rule>
</tableRule>

Function Definition

<function name="range-long" class="io.mycat.route.function.AutoPartitionByLong">
    <property name="mapFile">range-partition.txt</property>
</function>

Continuous Sharding Methods

Numeric Range Sharding

Partitions data based on predefined numeric ranges in configuraiton files.

# range start-end , node index
0-50000=0
50001-100000=1
100001-200000=2

Date-Based Sharding

Distributes data across partitions using date intervals.

<function name="date-partition" class="io.mycat.route.function.PartitionByDate">
    <property name="dateFormat">yyyy-MM-dd</property>
    <property name="startDate">2023-01-01</property>
    <property name="partitionDays">30</property>
</function>

Discrete Sharding Approaches

Enumeration Sharding

Maps specific values to designated partitions using configuration files.

# value = node index
100=0
200=1
300=2

Modulo Sharding

Distributes data using modulo arithmetic on numeric values.

<function name="mod-partition" class="io.mycat.route.function.PartitionByMod">
    <property name="count">4</property>
</function>

Consistent Hashing

Uses murmur hash algorithm for improved data distribution during scaling.

<function name="consistent-hash" class="io.mycat.route.function.PartitionByMurmurHash">
    <property name="seed">0</property>
    <property name="count">3</property>
    <property name="virtualBucketTimes">160</property>
</function>

Composite Sharding Strategies

Range-Modulo Combination

Combines range partitioning with modulo distribution within ranges.

# range start-end, modulo divisor
0-1000000=4
1000001-2000000=2

Date Range with Hash

Groups data by date ranges and applies hashing within each group.

<function name="date-range-hash" class="io.mycat.route.function.PartitionByRangeDateHash">
    <property name="dateFormat">yyyy-MM-dd HH:mm:ss</property>
    <property name="startDate">2023-01-01 00:00:00</property>
    <property name="rangeDays">30</property>
    <property name="groupSize">5</property>
</function>

Sharding Strategy Considerations

  • Evaluate table size - target ~10M rows per shard after partitioning
  • Select sharding key based on most frequent query patterns
  • Consiedr data migration requirements during scaling operations
  • Apply consistent sharding rules to related tables for join operations
  • Use global tables for rarely updated configuration data

Related Articles

Understanding Strong and Weak References in Java

Strong References Strong reference are the most prevalent type of object referencing in Java. When an object has a strong reference pointing to it, the garbage collector will not reclaim its memory. F...

Comprehensive Guide to SSTI Explained with Payload Bypass Techniques

Introduction Server-Side Template Injection (SSTI) is a vulnerability in web applications where user input is improper handled within the template engine and executed on the server. This exploit can r...

Implement Image Upload Functionality for Django Integrated TinyMCE Editor

Django’s Admin panel is highly user-friendly, and pairing it with TinyMCE, an effective rich text editor, simplifies content management significantly. Combining the two is particular useful for bloggi...

Leave a Comment

Anonymous

◎Feel free to join the discussion and share your thoughts.