Migrate AFFiNE specified server codes into AFFiNE monorepo

[Brooooooklyn]

Motivation

OctoBase contains too much AFFiNE business-related logic, which makes the scope of OctoBase very unclear. And it also makes it difficult for external developers to easily get started with OctoBase and contribute to it.
Then, using Rust to develop CRUD was very inefficient, and we wanted to be able to develop the business layer logic in Node.js and keep shipping fast.

Scope

• lib0 encoding/decoding for yjs binary Port lib0 encoding/decoding #383
• Port Subset of yjs
  • Subdocs
  • ...
• High-level features top on yjs doc
  • Search
  • Blob persists abstract layer
  • Database integration
    • yjs Doc Sync
    • yjs binary persist
    • Database extensions like sqlite-crdt or prosgres-crdt
  • Vector database integration like datafusion
• Languages and framework bindings
  • Node.js
    • TypeScript SDK
    • Nestjs integration
    • ...
  • Swift
  • Kotlin
  • ...

Migration path

Beta Give feedback

1. Bootstrap Node.js server in AFFiNE AFFiNE#1921
   4 of 4
   roadmap type:refactor +2
   
2. Migrate ORM layer
3. Glue Rust server with Node.js. This allows AFFiNE developers to migrate smoothly to Node.js locally
4. Retire AFFiNE specified server codes

[affine-bot]

Hello! Thank you for filing an issue.
If this is a bug report, please include relevant logs to help us debug the problem.

[darkskygit]

i think it is reasonable to migrate curd to nodejs
I think there are two feature that should also be clear included in the scope of octobase:

• storage: yjs cannot store binary at a reasonable cost, and ydoc also needs to be stored somewhere as a binary. storage is something that any application needs to consider in development
• sync: sync protocol impl should be included in octobase to avoid repeated implementation in different clients

[Brooooooklyn]

@darkskygit updated its into Database integration

[JasperStardream]

I think the sub-doc sync and data upgrade may need consideration in future.