Fundamental Concepts

What are Raw Devices, Character Devices, and Block Devices?

A raw device (or raw partition) is a special character device that has not been formatted and is not accessed through the file system by the Unix/Linux kernel. Applications interact directly with these devices for read/write operations, bypassing the system buffer cache. A raw device can be mapped to an entire disk or a specific partition.

A character device allows I/O operations without using the operating system's buffer cache. These devices cannot be mounted via a file system.

A block device relies on the OS buffer cache for read/write operations and can be mounted into the file system hierarchy.

In Linux, raw device support is not enabled by default; it requires manual configuraton of the raw service and adjustment of access permissions.

Benefits of Raw Devices for Oracle Databases

Using raw devices eliminates the overhead of the Unix/Linux file system layer, allowing data to transfer directly between the disk and Oracle. For databases with extremely high I/O throughput where disk I/O is the primary bottleneck, this can significantly boost performance—potentially by up to 40%. Additionally, since raw partitions bypass standard file system management, overheads related to inode maintenance, free block tracking, and file system journaling are removed.

Limits on Raw Device Counts

The maximum number of raw devices depends on the Linux kernel version:

• Older versions supported up to 256 raw devices.
• Linux 4.x and later can support up to 8192 raw bindings.

However, if binding to standard disk partitions, the limit is 255 partitions per disk. Using LVM (Logical Volume Manager) removes this partition-based limitation.

Partition Limits on a Single Disk

Linux allows a maximum of 15 partitions per disk:

• 3 Primary partitions
• 1 Extended partition
• 11 Logical partitions (inside the extended partition)

It is recommended to avoid binding raw devices to extended partitions.

Configuring Raw Device Bindings

Temporary Binding (Command Line)

You can manually bind a block device to a raw device node using the raw command. These bindings are lost after a reboot.

# Bind /dev/sdb1 to raw device 1
raw /dev/raw/raw1 /dev/sdb1
# Output: /dev/raw/raw1:  bound to major 8, minor 17

# Unbind the device
raw /dev/raw/raw1 0 0

# List all bindings
raw -qa
ls -l /dev/raw/

Permanent Binding (Configuration Files)

CentOS 6 and Older

Edit the /etc/udev/rules.d/60-raw.rules file to persist bindings across reboots.

# Example: Binding by Major/Minor number
ACTION=="add", ENV{MAJOR}=="252", ENV{MINOR}=="17", RUN+="/bin/raw /dev/raw/raw1 %M %m"

# Set permissions
ACTION=="add", KERNEL=="raw1", OWNER="grid", GROUP="asmadmin", MODE="660"

Restart the udev service to apply:

start_udev

CentOS 7 and Newer

The start_udev command is deprecated. Instead, use udevadm to reload rules.

# Reload rules and trigger changes
sudo udevadm control --reload-rules
sudo udevadm trigger --type=devices --action=change

# Alternatively, restart the trigger service
sudo systemctl restart systemd-udev-trigger.service

Identifying Disks via SCSI ID (UDEV)

For consistent naming (e.g., asm-diskb), use SCSI IDs to identify disks regardless of their /dev/sdX letter.

# Ensure scsi_id config is set
echo "options=--whitelisted --replace-whitespace" >> /etc/scsi_id.config

# Generate rules for disks (example for sdb, sdc, sdd)
for disk in b c d; do
  scsi_id=$(/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/sd$disk)
  echo "KERNEL==\"sd*\", BUS==\"scsi\", PROGRAM==\"/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/\$name\", RESULT==\"$scsi_id\", NAME=\"asm-disk$disk\", OWNER=\"grid\", GROUP=\"asmadmin\", MODE=\"0660\""
done

Place the generated output into /etc/udev/rules.d/99-oracle-asmdevices.rules.

Using Raw Devices with Oracle

Preparation

Ensure the raw device permissions are correct (e.g., owned by oracle:dba). This can be automated in /etc/rc.local if not using UDEV rules.

Constraints and Best Practices

1. One-to-One Mapping: A single raw device must contain exactly one data file.
2. Sizing: The data file size must not exceed the raw device size.
   • Redo Logs: Max size = (Partition Size) - (1 * 512 bytes).
   • Data Files: Max size = (Partition Size) - (2 * DB_BLOCK_SIZE).
   • General Rule: Leave atleast 1MB free on the raw device to be safe.
3. Auto-extend: If enabling auto-extend, set a MAXSIZE strictly smaller than the raw device capacity.

Logical Volumes (LVM)

Linux requires binding LVM logical volumes to raw devices explicitly; Oracle cannot use them directly as raw devices (unlike some Unix systems). The binding process is identical to partitioning.

Managing Raw Devices

Checking Size

To determine the size of a raw device, use blockdev:

blockdev --getsize /dev/raw/raw1
# Returns size in 512-byte sectors
# Example: 11718750 sectors * 512 bytes = ~5.7GB

Clearing Data

To wipe a raw device (similar to formatting), use dd:

# Overwrite with zeros (ensure count * bs > device size)
dd if=/dev/zero of=/dev/raw/raw4 bs=8192 count=128000

Unbinding

Set the major and minor numbers to 0:

raw /dev/raw/raw1 0 0

Common Utilities and UDEV Reference

Disk Information

# List partitions and block devices
cat /proc/partitions

# List block devices with UUIDs
lsblk -f

# SCSI device info
lsscsi --scsi_id

UDEV Directories

• /sys/class/block/: Kernel block device information.
• /dev/raw/: Raw device nodes.
• /dev/disk/by-id/: Persistent disk identifiers.
• /dev/disk/by-uuid/: UUID-based links.

Key UDEV Rule Parameters

Key	Description
ACTION	Event type (add, remove).
KERNEL	Device name (e.g., sd*).
SUBSYSTEM	Device subsystem (e.g., block).
ENV{MAJOR}	Major number identifying device type.
ENV{MINOR}	Minor number identifying partition/volume.
RUN	Command to execute.
OWNER/GROUP	File ownership.
MODE	File permissions (e.g., 0660).

Substitution Operators

• %N: Kernel name (用于临时节点).
• %M: Major number.
• %m: Minor number.
• %c: Result from a PROGRAM execution.

Mounting by UUID

To mount a filesystem using its UUID:

1. Find the UUID: blkid or lsblk -f.
2. Add to /etc/fstab:

   echo "UUID=xxxxxxxxx /mnt ext4 defaults 0 0" >> /etc/fstab
   

3. Test: mount -a.