This project demonstrates a pet cat detection system built around the Seeed Studio XIAO ESP32 S3 Sense board. The solution leverages computer vision techniques and embedded systems to monitor and analyze feline behavior in real time.

The hardware setup includes the XIAO ESP32 S3 Sense development board paired with an OV2640 camera module for capturing visual data. The collected images are processed using the RoboFlow platform for labeling and preparation. A custom-trained model is then generated via the AI Studio platform and deployed onto the device through the SenseCraft interface.

Key components:

• Hardware: Seeed Studio XIAO ESP32 S3 Sense
• Software Tools: Arduino IDE, RoboFlow, AI Studio, SenseCraft

The system performs real-time detection of cats within the monitored area. When a cat is detected, the board triggers an LED indicator to signal its presence.

Implementation Steps

1. Image Collection: Using the Arduino IDE, capture photos of cats and store them in a dataset.

2. Data Labeling: Upload the images to RoboFlow for annotation and export in COCO format.

3. Model Training: Use AI Studio’s ModelAssistant framework to train a Swift-YOLO-based model.

   !git clone https://github.com/Seeed-Studio/ModelAssistant.git
   %cd ModelAssistant
   
   %env DATA_ROOT="https://universe.roboflow.com/ds/aliqVGka4t?key=JibV8Iog4S"
   %env NUM_CLASSES=1
   
   !python tools/train.py \
     configs/swift_yolo/swift_yolo_tiny_1xb16_300e_coco.py \
     --cfg-options \
       epochs=10 \
       num_classes=${NUM_CLASSES} \
       workers=1 \
       imgsz=192,192 \
       data_root=${DATA_ROOT} \
       load_from=https://files.seeedstudio.com/sscma/model_zoo/detection/person/person_detection.pth
   

4. Deployment: Upload the trained model to the SenseCraft platform for deployment on the XIAO board.

Code Snippet for Image Capture

Initialization checks ensure proper setup before proceeding:

if(camera_sign && sd_sign){
  String command;
  while (Serial.available()) {
    char c = Serial.read();
    if ((c != '\n') && (c != '\r')) {
      command.concat(c);
    } else if (c == '\n') {
      commandRecv = true;
      command.toLowerCase();
    }
  }

  if (commandRecv && command == "capture") {
    commandRecv = false;
    Serial.println("\nPicture Capture Command is sent");
    char filename[32];
    sprintf(filename, "/image%d.jpg", imageCount);
    photo_save(filename);
    Serial.printf("Saved picture：%s\n", filename);
    Serial.println("");
    imageCount++;
  }
}

Saving captured frames to SD card:

void photo_save(const char *filename) {
  camera_fb_t *fb = esp_camera_fb_get();
  if (!fb) {
    Serial.println("Failed to get camera frame buffer");
    return;
  }
  writeFile(SD_MMC, filename, fb->buf, fb->len);
}

void writeFile(fs::FS &fs, const char *path, uint8_t *data, size_t len) {
  Serial.printf("Writing file: %s\n", path);
  File file = fs.open(path, FILE_WRITE);
  if (!file) {
    Serial.println("Failed to open file for writing");
    return;
  }
  if (file.write(data, len) == len) {
    Serial.println("File written");
  } else {
    Serial.println("Write failed");
  }
  file.close();
}

Challenges Encountered

1. Low Recognition Accuracy: Initial training with a small dataset (<200 images) led to poor accuracy. Increasing the dataset size improved performance.

2. Pin Control Limitations: During operation, only one LED action could be executed; additional pin configurations were not supported.

3. Overheating: The chip became excessively warm during continuous operation.

4. Low Frame Rate: Camera output was limited to 10 FPS, affecting recognition responsiveness.

Future Improvements

1. Enhanced Hardware: Replace the OV2640 with a higher-resolution camera for better clarity.

2. Extended Functionality: Integrate motor drivers and sensors to support smart home automation features.

3. Performance Upgrade: Consider a more powerful microcontroller to increase frame rate.

4. Model Optimization: Train with larger datasets and expand labels to detect various pet types.

5. Remote Access: Enable network connectivity for remote monitoring and control.

6. Bluetooth Integration: Add Bluetooth capability for connecting peripheral devices.