Overview of Inter-Process Communication

IPC Objectives

• Data Transfer: Send data from one process to another
• Resource Sharing: Allow multiple processes to access shared resources
• Event Notification: Alert one or more processes of system or application events (e.g., a parent process receiving notification when a child process exits)
• Process Control: Fully control another process's execusion (such as debuggers intercepting exceptions and tracking state changes of target processes)

Evolution of IPC Mechanisms

1. Pipes
2. System V Inter-Process Communication
3. POSIX Inter-Process Communication

IPC Classification

• Pipes
  • Anonymous Pipes
  • Named Pipes (FIFOs)
• System V IPC
  • System V Message Queues
  • System V Shared Memory
  • System V Semaphores
• POSIX IPC
  • Message Queues
  • Shared Memory
  • Semaphores
  • Mutexes
  • Condition Variables
  • Read-Write Locks

---

1. Pipes

1.1 Basic Pipe Concepts

Pipes are the oldest form of IPC on Unix-like systems. A pipe is a unidirectional data stream connecting two processes.

Anonymous pipes are limited to communicating between related processes (e.g., parent and child processes) and are created using the pipe system call. Key traits of anonymous pipes:

1. They are implemented as a kernel buffer, treated as a pseudo-file in the kernel
2. Accessed via two file descriptors: one for read operations, one for write operations
3. Data flows from the write end to the read end

The underlying implementation uses a kernel circular buffer (typically 4KB in size).

Pipe Characteristics

• Only usable between processes with a common ancestor (related processes)
• Provides streaming service
• Lifecycle tied to the processes using it: the pipe is released when all participating processes exit
• Kernel enforces synchronization and mutual exclusion for pipe operations
• Half-duplex: data travels in one direction only; bidirectional communication requires two separate pipes

Pipe Limitations

• A process cannot read and write to its own pipe
• Data is removed from the pipe once read, and cannot be re-read
• Only supports unidirectional traffic without a second pipe
• Restricted to related processes

Pipe Read/Write Rules

1. No data available:
   • With O_NONBLOCK disabled: read blocks until data is available
   • With O_NONBLOCK enabled: read returns -1 with errno set to EAGAIN
2. Pipe full:
   • With O_NONBLOCK disabled: write blocks until data is read from the pipe
   • With O_NONBLOCK enabled: write returns -1 with errno set to EAGAIN
3. Closed file descriptors:
   • If the write end is closed: read returns 0 after consuming all remaining data, no longer blocks
   • If the read end is closed: write triggers a SIGPIPE signal, which may terminate the writing process
4. Atomicity:
   • Writes smaller than or equal to PIPE_BUF are guaranteed atomic
   • Writes larger than PIPE_BUF do not guarantee atomicity
5. Additional Notes:
   • Pipe lifecycle is tied to processes. Named pipe files act as identifiers for processes to locate the kernel buffer; deleting the named pipe file does not break existing active connections

1.2 Creating Pipes

Anonymous Pipes

Use the pipe system call:

#include <unistd.h>
int pipe(int pipe_fds[2]);

Function Details:

• Creates an anonymous pipe
• pipe_fds: Two-element array storing file descriptors:
  • pipe_fds[0]: Read end file descriptor
  • pipe_fds[1]: Write end file descriptor
• Return values:
  • 0 on success
  • -1 on failure, with errno set appropriately

Example Implementation:

#include <iostream>
#include <cassert>
#include <unistd.h>
#include <cstdio>
#include <cstring>

int main() {
    int pipe_fds[2] = {0};
    int ret = pipe(pipe_fds);
    assert(ret == 0);
    static_cast<void>(ret); // Suppress unused variable warning in release builds
    
    std::cout << "pipe_fds[0]: " << pipe_fds[0] << ", pipe_fds[1]: " << pipe_fds[1] << std::endl;
    return 0;
}

Workflow for Inter-Process Communication:

1. Parent process calls pipe to create the pipe
2. Parent process uses fork to create a child process, which inherits the pipe file descriptors
3. Parent and child close unused pipe ends to enable unidirectional communication

Named Pipes (FIFOs)

Named pipes allow communication between unrelated processes, acting as special file types visible in the filesystem (identified by a p prefix when listed with ls -l).

Creating Named Pipes:

1. Command Line:

   mkfifo my_fifo
   

2. Programmatic:

   #include <sys/stat.h>
   #include <iostream>
   
   int main() {
       if (mkfifo("server_pipe", 0644) == -1) {
           std::perror("mkfifo failed");
           return 1;
       }
       return 0;
   }
   

Function Details:

• mkfifo(const char* filename, mode_t mode)
  • filename: Path to the named pipe file
  • mode: File permission bits (same as standard file permissions)
• Return values:
  • 0 on success
  • -1 on failure; EEXIST indicates the pipe file already exists

Blocking Behavior:

• Opening a named pipe for read-only blocks until another process opens it for write-only
• Opening a named pipe for write-only blocks until another process opens it for read-only

Anonymous vs Named Pipes

• Anonymous pipes are created and opened with pipe
• Named pipes are created with mkfifo and opened with open
• Both have identical semantics once set up, differing only in creation and initial opening steps

1.3 Deleting Pipes

Use standard filesystem commands to remove named pipe files:

unlink my_fifo
rm -f my_fifo

1.4 Four Special Pipe Scenarios

1. Reader waits for data: Reader blocks until data is written to the pipe
2. Writer blocks when pipe full: Writer blocks until data is read from the pipe
3. Writer closes after sending all data: Reader consumes remaining data and exits without blocking
4. Reader closes while writer continues: Kernel terminates the writing process to avoid sending data to a closed reader

---

2. System V IPC

System V IPC is a kernel-designed communication mechanism, unlike pipes which use the filesystem layer. All System V IPC mechanisms rely on allowing multiple processes to access a shared kernel resource.

The three System V IPC types are:

• Shared Memory: Fastest IPC mechanism, no kernel data copies after initial mapping
• Message Queues: Ordered message passing between processes
• Semaphores: Synchronization primitives to manage access to shared resources

---

2.1 Shared Memory (SHM)

Basic Concepts

Shared memory is a region of kernel memory mapped directly into the address spaces of participating processes. The kernel manages the memory region, and all processes can read/write directly to it without syscall overhead after the initial mapping.

Advantages: Faster than pipes, as data does not require kernel copies for transfer; supports communication between unrelated processes. Disadvantages: No built-in synchronization; requires external tools (like semaphores) to coordinate access between reader and writer processes.

Key Notes:

• Shared memory persists until explicitly deleted or the system reboots
• Deletion marks the memory for removal only when all attached processes detach

Shared Memory Functions

All functions require these headers:

#include <sys/ipc.h>
#include <sys/shm.h>

ftok - Generate IPC Key

Converts a existing file path and integer ID into a unique key_t value for IPC objects:

#include <sys/types.h>
#include <sys/ipc.h>

key_t ftok(const char* pathname, int proj_id);

• pathname: Path to an existing, accessible file
• proj_id: Low-order 8 bits used to generate the key
• Returns key_t on success, -1 on failure

shmget - Create/Access Shared Memory

int shmget(key_t ipc_key, size_t size, int flags);

• ipc_key: Unique IPC key generated by ftok
• size: Size of the shared memory segment, ideally a multiple of 4096
• flags: Permission bits and creation flags:
  • IPC_CREAT: Create segment if it does not exist
  • IPC_EXCL: Fail if segment already exists (used with IPC_CREAT)
  • Combined with permission bits (e.g., 0664)
• Returns non-negative segment ID on success, -1 on failure

Example:

key_t ipc_key = ftok("/home/user/linux_project", 0x42);
int shm_id = shmget(ipc_key, 4096, IPC_CREAT | IPC_EXCL | 0664);

shmat - Attach Shared Memory to Process Address Space

void* shmat(int shm_id, const void* addr, int flags);

• shm_id: Shared memory segment ID from shmget
• addr: Desired attachment address; pass nullptr to let the kernel choose automatically
• flags: Set to 0 for standard behavior
• Returns pointer to attached memory on success, (void*)-1 on failure

shmdt - Detach Shared Memory

int shmdt(const void* shm_ptr);

• shm_ptr: Pointer returned by shmat
• Returns 0 on success, -1 on failure

Note: Detaching a shared memory segment does not delete it

shmctl - Control Shared Memory

int shmctl(int shm_id, int cmd, struct shmid_ds* buf);

• shm_id: Shared memory segment ID
• cmd: Action to perform:
  • IPC_STAT: Get segment status
  • IPC_SET: Modify segment permissions
  • IPC_RMID: Mark segment for deletion
• buf: Pointer to shmid_ds structure for status/settings; pass nullptr for deletion
• Returns 0 on success, -1 on failure

Example Deletion:

shmctl(shm_id, IPC_RMID, nullptr);

---

2.2 Message Queues

Basic Concepts

Message queues are linked lists of messages stored in the kernel, identified by a unique ID. Proceses send and receive messsages with specific types, avoiding the ordering limitations of pipes.

Limitations: Fixed maximum message size, maximum total bytes per queue, and system-wide maximum queue count.

Message Queue Functions

Required headers:

#include <sys/types.h>
#include <sys/ipc.h>
#include <sys/msg.h>

msgget - Create/Access Message Queue

int msgget(key_t ipc_key, int flags);

• ipc_key: Unique IPC key from ftok
• flags: Permission bits and creation flags
• Returns non-negative queue ID on success, -1 on failure

msgsnd - Send Message to Queue

int msgsnd(int msq_id, const void* msg_ptr, size_t msg_size, int flags);

// Standard message structure
struct custom_msg {
    long msg_type;
    char msg_data[256];
};

• msq_id: Message queue ID
• msg_ptr: Pointer to message structure
• msg_size: Size of message data (excluding the msg_type long integer)
• flags: Set to 0 for blocking behavior
• Returns 0 on success, -1 on failure

Example:

int send_message(int msq_id, long msg_type, const char* input_data) {
    custom_msg send_buf;
    send_buf.msg_type = msg_type;
    strncpy(send_buf.msg_data, input_data, sizeof(send_buf.msg_data)-1);
    send_buf.msg_data[sizeof(send_buf.msg_data)-1] = '\0';
    
    if (msgsnd(msq_id, &send_buf, sizeof(send_buf.msg_data), 0) == -1) {
        std::perror("msgsnd failed");
        return -1;
    }
    return 0;
}

msgrcv - Receive Message from Queue

ssize_t msgrcv(int msq_id, void* msg_ptr, size_t msg_size, long msg_type, int flags);

• msq_id: Message queue ID
• msg_ptr: Buffer to store received message
• msg_size: Size of message data buffer
• msg_type: Type of message to receive
• flags: Set to 0 for blocking behavior
• Returns number of bytes received on success, -1 on failure

msgctl - Control Message Queue

int msgctl(int msq_id, int cmd, struct msqid_ds* buf);

• msq_id: Message queue ID
• cmd: Action to perform (e.g., IPC_RMID to delete queue)
• buf: Pointer to status structure, pass nullptr for deletion
• Returns 0 on success, -1 on failure

---

2.3 Semaphores

Semaphores are synchronization primitives used to coordinate access to shared resources. They support atomic P (wait/lock) and V (post/unlock) operations.

Semaphore Functions

semget - Create/Access Semaphore Set

int semget(key_t ipc_key, int num_sems, int flags);

• ipc_key: Unique IPC key
• num_sems: Number of semaphores in the set (almost always 1 for single-resource control)
• flags: Permission bits and creation flags
• Returns non-negative semaphore set ID on success, -1 on failure

semop - Perform Semaphore Operations

int semop(int sem_id, struct sembuf* ops, size_t num_ops);

struct sembuf {
    short sem_num; // Index of semaphore in the set
    short sem_op;  // Operation: -1 for P, +1 for V
    short sem_flags; // Set to SEM_UNDO for automatic cleanup on process exit
};

• sem_id: Semaphore set ID
• ops: Array of sembuf structures defining operations
• num_ops: Number of operations in the array

semctl - Control Semaphores

int semctl(int sem_id, int sem_num, int cmd, ...);

union semun {
    int val; // Value for SETVAL
    struct semid_ds* buf;
    unsigned short* array;
};

• sem_id: Semaphore set ID
• sem_num: Index of semaphore in the set
• cmd: Action to perform:
  • SETVAL: Initialize semaphore value
  • IPC_RMID: Delete semaphore set
• Fourth optional argument is a union semun for initialization

---

3. Command Line IPC Management

View IPC Resources

# List shared memory segments
ipcs -m
# List message queues
ipcs -q
# List semaphore sets
ipcs -s

• nattch field shows number of processes attached to the shared memory segment

Delete IPC Resources

# Delete shared memory segment
ipcrm -m [shm_id]
# Delete message queue
ipcrm -q [msq_id]
# Delete semaphore set
ipcrm -s [sem_id]

# Delete all IPC resources
ipcrm -a

---

4. Kernel IPC Data Structures

The kernel maintains metadata structures for all IPC objects:

struct ipc_ids {
    int size; // Size of entries array
    int in_use; // Number of active entries
    int max_id;
    unsigned short seq;
    unsigned short seq_max;
    struct semaphore sem; // Lock for IPC object access
    spinlock_t ary; // Spinlock for entries array
    struct ipc_id* entries; // Flexible array of IPC object pointers
};

struct ipc_id {
    struct ipc_perm *p;
};

struct ipc_perm {
    key_t __key; // IPC key from ftok
    uid_t uid; // Owner user ID
    gid_t gid; // Owner group ID
    uid_t cuid; // Creator user ID
    gid_t cgid; // Creator group ID
    unsigned short mode; // Permission bits
    unsigned short __seq; // Sequence number
};

// Shared memory metadata
struct shmid_ds {
    struct ipc_perm shm_perm;
    size_t shm_segsz;
    time_t shm_atime;
    time_t shm_dtime;
    time_t shm_ctime;
    pid_t shm_cpid;
    pid_t shm_lpid;
    shmatt_t shm_nattch;
};

// Message queue metadata
struct msqid_ds {
    struct ipc_perm msg_perm;
    time_t msg_stime;
    time_t msg_rtime;
    time_t msg_ctime;
    unsigned long __msg_cbytes;
    msgqnum_t msg_qnum;
    msglen_t msg_qbytes;
    pid_t msg_lspid;
    pid_t msg_lrpid;
};

// Semaphore set metadata
struct semid_ds {
    struct ipc_perm sem_perm;
    time_t sem_otime;
    time_t sem_ctime;
    unsigned long sem_nsems;
};