Blog

Sector
Neel Patel posted on Jul 15, 2011
  • What is it?
    • Sector stores files on the native/local file system of each slave node.
    • Sector does not split files into blocks
      • Pro: simple/robust, suitable for wide area
      • Con: file size limit
    • Sector uses replications for better reliability and availability
      The master node maintains the file system metadata. No permanent metadata is needed.
    • Topology aware
  • Read/Write
    • Write is exclusive
    • Replicas are updated in a chained manner: the client updates one replica, and then this replica updates another, and so on. All replicas are updated upon the completion of a Write operation.
    • Read: different replicas can serve different clients at the same time. Nearest replica to the client is chosen whenever possible.
  • Tools/API
    • Supported file system operation: ls, stat, mv, cp, mkdir, rm, upload, download
      Wild card characters supported
    • System monitoring: sysinfo.
    • C++ API: list, stat, move, copy, mkdir, remove, open, close, read, write, sysinfo.