Merge main to saber (#1)

* Use accurate token counts (BloopAI#1024)

* Use accurate token counts

The `total` token count is now based upon an identical format
conversation sent to the LLM during response generation.

This results in counts that should be accurate, and prevent token limit
errors entirely.

* Match token counts precisely, and add baseline count

* Calculate `total` as summation of other section token counts

* add message to response if it ended when token limit exceeded, no translations

---------

Co-authored-by: anastasiia <[email protected]>

* Send full redirect target to cognito (BloopAI#1034)

* Update Helm chart (BloopAI#1033)

* update helm chart

* remove secret.yaml

* put back secret.yaml

* mandatory Semantic in /search (BloopAI#1039)

* Upgrade qdrant version to 1.6 (BloopAI#1037)

* Upgrade qdrant version

* Update qdrant binaries

* Fix race in credentials polling (BloopAI#1042)

Previously the assumption here was that this path is locked & safe
when there is a furnished github cred in the system. However, when the
user logs out, the `unwrap()` call can blow up the task, and
this may cause issues.

* Enable paid features for desktop users (BloopAI#1038)

* Add pro features to default builds

* Check user's status through `User` object

* Adapt webserver layer for paid feature gate

* Just enforce schema right out of the gate

* Fix date parsing logic

* allow paid users sync branches

* Fix clippy features

* Disable branch switching for local repos

* cloud implies pro

* Update dockerfile

---------

Co-authored-by: anastasiia <[email protected]>

* Update flake (BloopAI#1044)

* Debug logs for initialisation (BloopAI#1036)

* debug logs for initialisation

* Scope logging

* DB too

* Clippy error

---------

Co-authored-by: rsdy <[email protected]>

* Allow indexing of large files (BloopAI#1040)

* WIP

* Support indexing large files, but lazy load them from local file system

* bump tantivy to v0.21 (BloopAI#1043)

* bump tantivy to v0.22

* address clippy

* fix broken tests

* bump version to 0.5.6 (BloopAI#1047)

---------

Co-authored-by: calyptobai <[email protected]>
Co-authored-by: anastasiia <[email protected]>
Co-authored-by: rsdy <[email protected]>
Co-authored-by: Gabriel Gordon-Hall <[email protected]>
Co-authored-by: rsdy <[email protected]>
Co-authored-by: akshay <[email protected]>
Co-authored-by: Anastasiia Solop <[email protected]>
Co-authored-by: Ilya Zedgenizov <[email protected]>

.github/workflows/dependencies.yml

@@ -24,15 +24,15 @@ jobs:
 
         include:	
           - target: x86_64-unknown-linux-gnu	
-            os: ubuntu-latest	
-            cross: false	
+            os: ubuntu-latest
+            cross: false
 
           - target: x86_64-apple-darwin	
-            os: macos-11	
-            cross: false	
+            os: macos-11
+            cross: false
 
           - target: aarch64-apple-darwin	
-            os: macos-11	
+            os: macos-11
             cross: true
 
           - target: x86_64-pc-windows-msvc
@@ -44,7 +44,7 @@ jobs:
       - name: Install Rust stable	
         uses: dtolnay/rust-toolchain@stable
         with:	
-          toolchain: stable	
+          toolchain: 1.70.0
           target: ${{ matrix.target }}	
 
       - name: Install Protoc
@@ -54,9 +54,9 @@ jobs:
 
       - name: Build qdrant	
         env:	
-          VERSION: 1.5.1
+          VERSION: 1.6.0
         run: |	
-          cargo install --target ${{ matrix.target }} --git https://github.com/qdrant/qdrant --tag v${{ env.VERSION }} --locked --root . qdrant	
+          cargo install --target ${{ matrix.target }} --git https://github.com/qdrant/qdrant --tag v${{ env.VERSION }} --root . qdrant	
       - name: Upload binaries	
         uses: actions/upload-artifact@v3	
         with:	

.github/workflows/server-test.yml

@@ -4,6 +4,7 @@ on:
   pull_request:
     branches: [main]
     paths:
+      - "flake.*"
       - "server/**"
       - ".github/workflows/server**"
 
@@ -49,7 +50,7 @@ jobs:
         run: cargo --locked fmt -p bleep -- --check
 
       - name: Clippy
-        run: cargo --locked clippy -p bleep --features=ee
+        run: cargo --locked clippy -p bleep --features=ee-pro,ee-cloud
 
       - name: Tests
         run: cargo --locked test -p bleep --release