[Feature] Data Science Grouping (#689)

* Add empty DSG page & nav (#471) * Add empty DSG page & nav * Rework the way navs work & add feature flag * Kubernetes Pass Through API & SDK Initialization (#279) * SDK npm installed * Pass-through api & SDK setup * Migrated ConfigMap, Secret, PVC to SDK calls * Adjust k8s pass through to handle bad create urls * Code cleanup * Add local dev token & header access token support * Remove v12 from the supported node-versions * Fix test issue & convert starting notebook to SDK call * Added some docs and fixed a type issue * Address a couple issues with the flow * Improve server logging * Misc cleanup of DSG (#481) * Add project view details page (#578) * Add project view details page * address comment * make jumplink items generic * modify type * small fixes * Create projects (#597) * Create projects * Added workaround for self signed certs * List Projects & Delete Project (#605) * List of Projects with Notebook Details * Make Notebooks have display annotations * Pagination and filter * Handle pagination edge cases * Edit projects * Improve edit modal data management * Delete project * Add an filterd empty row * Cleaned up projects to be more organized & fixed issue with 404 project * Add a watcher for when starting the notebook via toggle * Add bad certif workaround to debug script * Create add storage modal (#582) * Create add storage modal * refactor files * clean up import * fix layout of existing storage * pass less props * address comments * Add part of spawner page (#610) * Add part of spawner page * add creating notebook * fix small issues * Add storage section for spawner page (#637) * Workspace Table (#660) * Follow up fix for #610 and #637 (#664) * follow up fix for #610 * follow up PR #637 * clean up * Code cleanup (#665) * Cleanup of Spawner code * Update the image details in the workspaces table * Add storage list layout (#666) * Add storage list layout * address comments * make JSX and functions inline * Data Connections List (#671) * Stop Notebook Confirmation Modal (#672) * Delete Notebook Modal (#673) * Delete Notebook Modal * Refresh only if finished deleting * Spawner Environment Section (#670) * Added spawner page. Refactor code part 1. Refactor secret and config map. Added upload code. Fixed style issues. * Refactored code. * Disabled AWS. * Updated pass through code. * Fixed linting errors. * Removed some dead code. AWS update. updates to secret Reverted code. * Fixed linting errors. * Delete Storage Modal (#677) * Prometheus connection for inUse storage value (#676) * Added spawner page. (#679) Refactor code part 1. Refactor secret and config map. Added upload code. Fixed style issues. Refactored code. Disabled AWS. Updated pass through code. Fixed linting errors. Removed some dead code. AWS update. updates to secret Added AWS code. Fixed rebase files. Fixed linting errors. * Cleanup Random DSG stuff (#678) * Added breadcrumb for project details and spawner page. (#682) * Adjust Storage based on UX input (#684) * Add status tooltip and start notebook modal (#683) * Add status tooltip and start notebook modal * get projectName from notebook namespace * Environment Spawner Section & Data Connections (#688) * Cleanup lint issues * Create dedicated OAuthclient (#658) * DSG Fixes (#696) * Source all the details data from the same spot * Connected Workspaces now pull from context * Follow-up fix for DSG work (#695) * bug fixes for dsg * rebase * Refresh notebook status and fetch all status periodically (#697) * refresh notebook status and fetch all status periodically * small fix for listening notebook * some rename and clean up * add input group for mount path and change placeholder text * refactor useRelatedNotebooks * Improve storage section features and bug fix (#698) * use config pvs size and some small bug fix * storage part refactor * add check for terminated status * Add ability to delete attached PVC (#699) * Add ability to delete attached PVC * add inline for alert * Improve Project Creation & Data Connection Modal (#700) * Misc DSG Fixes * Projects creation flexibility & data connection functionality * Fixes for DSG (#701) * update storage name and description * add aria-label and id on spawner page and project details page * add data-label for table cells * add GPU field and remove secure route for backend GPU * add generic delete modal * Translate display names into k8s names (#702) * Translate display names into k8s names * Minor changes to text and support a more direct delete button text * Fix lint issue * Fixes and features for DSG (#704) * delete data connection will also delete envFrom field * change switch to separate text and switch button * indent data connection list * make data connection name sortable * sort notebooks by status * change select menuAppendTo on spawner page * add truncate for dependency package details * make table show expand style correctly * clean up dependencies * add label to all the resources created by dashboard * UX Feedback -- DSG Changes (#705) * Add icon & tooltip for k8s resource identification * Allow searching on user associated to Projects * Improved tooltips and added more to the project creation * Add generic padding to the bottom of the scrollable content * Cleanup delete modal as it closes * Updated UX wording * Edit Notebook * Realign tooltip -- removeFindDomNode issue * Fix lint issues Co-authored-by: Juntao Wang <[email protected]> Co-authored-by: Donald Labaj <[email protected]> Co-authored-by: Samu <[email protected]>
opendatahub-io · Oct 31, 2022 · e8ff744 · e8ff744
1 parent aca4f29
commit e8ff744
Show file tree

Hide file tree

Showing 192 changed files with 10,457 additions and 2,101 deletions.
diff --git a/.github/workflows/pull_request.yml b/.github/workflows/pull_request.yml
@@ -8,7 +8,7 @@ jobs:
     runs-on: ubuntu-latest
     strategy:
       matrix:
-        node-version: [12.x, 14.x]
+        node-version: [14.x]
     steps:
       - uses: actions/checkout@v3
       - name: Setup Node.js ${{ matrix.node-version }}

diff --git a/backend/package-lock.json b/backend/package-lock.json
diff --git a/backend/package.json b/backend/package.json
@@ -19,8 +19,8 @@
   "main": "src/server.ts",
   "scripts": {
     "start": "npm run tsc && NODE_ENV=production node ./dist/server.js --log=1 --registry=localhost:50051",
-    "start:dev": "export NODE_ENV=development && nodemon src/server.ts --log=1 --registry=localhost:50051",
-    "debug": "npm run tsc && export NODE_ENV=development && node --inspect ./dist/server.js --log=1 --registry=localhost:50051",
+    "start:dev": "export NODE_TLS_REJECT_UNAUTHORIZED=0 && export NODE_ENV=development && nodemon src/server.ts --log=1 --registry=localhost:50051",
+    "debug": "npm run tsc && export NODE_TLS_REJECT_UNAUTHORIZED=0 && export NODE_ENV=development && node --inspect ./dist/server.js --log=1 --registry=localhost:50051",
     "build-only": "tsc -p . && node ./dist/server.js --log=1 --registry=localhost:50051 --buildonly",
     "build": "npm run build:clean; npm run tsc",
     "build:clean": "rimraf ./dist",

diff --git a/backend/src/routes/api/cluster-settings/clusterSettingsUtils.ts b/backend/src/routes/api/cluster-settings/clusterSettingsUtils.ts
@@ -71,6 +71,9 @@ export const updateClusterSettings = async (
                 kind: 'ConfigMap',
                 metadata: {
                   name: 'notebook-controller-culler-config',
+                  labels: {
+                    'opendatahub.io/dashboard': 'true',
+                  },
                 },
                 data: {
                   ENABLE_CULLING: String(isEnabled),

diff --git a/backend/src/routes/api/gpu/index.ts b/backend/src/routes/api/gpu/index.ts
@@ -1,12 +1,8 @@
 import { KubeFastifyInstance } from '../../../types';
 import { getGPUNumber } from './gpuUtils';
-import { secureRoute } from '../../../utils/route-security';
 
 export default async (fastify: KubeFastifyInstance): Promise<void> => {
-  fastify.get(
-    '/',
-    secureRoute(fastify)(async () => {
-      return getGPUNumber(fastify);
-    }),
-  );
+  fastify.get('/', async () => {
+    return getGPUNumber(fastify);
+  });
 };
diff --git a/backend/src/routes/api/images/imageUtils.ts b/backend/src/routes/api/images/imageUtils.ts
@@ -211,6 +211,7 @@ export const postImage = async (
   const labels = {
     'app.kubernetes.io/created-by': 'byon',
     'opendatahub.io/notebook-image': 'true',
+    'opendatahub.io/dashboard': 'true',
   };
   const imageStreams = (await getImageStreams(fastify, labels)) as ImageStream[];
   const validName = imageStreams.filter((is) => is.metadata.name === body.name);

diff --git a/backend/src/routes/api/k8s/index.ts b/backend/src/routes/api/k8s/index.ts
@@ -0,0 +1,46 @@
+import { FastifyReply, FastifyRequest } from 'fastify';
+import { KubeFastifyInstance } from '../../../types';
+import { USER_ACCESS_TOKEN } from '../../../utils/constants';
+import { passThrough } from './pass-through';
+
+module.exports = async (fastify: KubeFastifyInstance) => {
+  const kc = fastify.kube.config;
+  const cluster = kc.getCurrentCluster();
+
+  /**
+   * Pass through API for all things kubernetes
+   * Acts on the user who made the call -- does not need route security; k8s provides that.
+   */
+  fastify.route({
+    method: ['DELETE', 'GET', 'HEAD', 'PATCH', 'POST', 'PUT', 'OPTIONS'],
+    url: '/*',
+    handler: (
+      req: FastifyRequest<{
+        Querystring: Record<string, string>;
+        Params: { '*': string; [key: string]: string };
+        Body: { [key: string]: unknown };
+        Headers: { [USER_ACCESS_TOKEN]: string };
+      }>,
+      reply: FastifyReply,
+    ) => {
+      const data = JSON.stringify(req.body);
+      const kubeUri = req.params['*'];
+      let url = `${cluster.server}/${kubeUri}`;
+
+      // Apply query params
+      const query = req.query;
+      if (Object.keys(query)) {
+        url += `?${Object.keys(query)
+          .map((k) => `${encodeURIComponent(k)}=${encodeURIComponent(query[k])}`)
+          .join('&')}`;
+      }
+
+      return passThrough(fastify, req, { url, method: req.method, requestData: data }).catch(
+        ({ code, response }) => {
+          reply.code(code);
+          reply.send(response);
+        },
+      );
+    },
+  });
+};
diff --git a/backend/src/routes/api/k8s/pass-through.ts b/backend/src/routes/api/k8s/pass-through.ts
@@ -0,0 +1,117 @@
+import { FastifyRequest } from 'fastify';
+import https, { RequestOptions } from 'https';
+import { K8sResourceCommon, K8sStatus, KubeFastifyInstance } from '../../../types';
+import { DEV_MODE, USER_ACCESS_TOKEN } from '../../../utils/constants';
+import { getDirectCallOptions } from '../../../utils/directCallUtils';
+
+type PassThroughData = {
+  method: string;
+  requestData: string;
+  url: string;
+};
+
+const isK8sStatus = (data: Record<string, unknown>): data is K8sStatus => data.kind === 'Status';
+
+const setupRequest = async (
+  fastify: KubeFastifyInstance,
+  request: FastifyRequest<{ Headers: { [USER_ACCESS_TOKEN]: string } }>,
+  data: PassThroughData,
+): Promise<{ url: string; requestOptions: RequestOptions }> => {
+  const { method, url } = data;
+
+  // TODO: Remove when bug is fixed - https://issues.redhat.com/browse/HAC-1825
+  let safeURL = url;
+  if (method.toLowerCase() === 'post') {
+    // Core SDK builds the wrong path for k8s -- can't post to a resource name; remove the name from the url
+    // eg: POST /.../configmaps/my-config-map => POST /.../configmaps
+    const urlParts = url.split('/');
+    urlParts.pop();
+    safeURL = urlParts.join('/');
+  }
+
+  const requestOptions = await getDirectCallOptions(fastify, request, url);
+
+  return {
+    url: safeURL,
+    requestOptions: {
+      ...requestOptions,
+      method,
+    },
+  };
+};
+
+export const passThrough = (
+  fastify: KubeFastifyInstance,
+  request: FastifyRequest<{ Headers: { [USER_ACCESS_TOKEN]: string } }>,
+  data: PassThroughData,
+): Promise<K8sResourceCommon> => {
+  const { method, requestData } = data;
+
+  return new Promise((resolve, reject) => {
+    setupRequest(fastify, request, data).then(({ url, requestOptions }) => {
+      if (requestData) {
+        requestOptions.headers = {
+          ...requestOptions.headers,
+          'Content-Type': `application/${
+            method === 'PATCH' ? 'json-patch+json' : 'json'
+          };charset=UTF-8`,
+          'Content-Length': requestData.length,
+        };
+      }
+
+      fastify.log.info(`Making API ${method} request to ${url}`);
+
+      const httpsRequest = https
+        .request(url, requestOptions, (res) => {
+          let data = '';
+          res
+            .setEncoding('utf8')
+            .on('data', (chunk) => {
+              data += chunk;
+            })
+            .on('end', () => {
+              let parsedData: K8sResourceCommon | K8sStatus;
+              try {
+                parsedData = JSON.parse(data);
+              } catch (e) {
+                // Likely not JSON, print the error and return the content to the client
+                fastify.log.error(`Parsing response error: ${e}, ${data}`);
+                reject({ code: 500, response: data });
+                return;
+              }
+
+              if (isK8sStatus(parsedData)) {
+                if (parsedData.status !== 'Success') {
+                  fastify.log.warn(
+                    `Unsuccessful status Object, ${
+                      DEV_MODE ? JSON.stringify(parsedData, null, 2) : JSON.stringify(parsedData)
+                    }`,
+                  );
+                  reject({ code: parsedData.code, response: parsedData });
+                  return;
+                }
+              }
+
+              fastify.log.info('Successful request, returning data to caller.');
+              resolve(parsedData);
+            })
+            .on('error', (error) => {
+              if (error) {
+                fastify.log.error(`Kube parsing response error: ${error}`);
+                reject({ code: 500, response: error });
+              }
+            });
+        })
+        .on('error', (error) => {
+          fastify.log.error(`Kube request error: ${error}`);
+          reject({ code: 500, response: error });
+        });
+
+      if (requestData) {
+        httpsRequest.write(requestData);
+      }
+
+      httpsRequest.end();
+    });
+  });
+};
diff --git a/backend/src/routes/api/prometheus/index.ts b/backend/src/routes/api/prometheus/index.ts
@@ -0,0 +1,19 @@
+import { KubeFastifyInstance, OauthFastifyRequest, PrometheusResponse } from '../../../types';
+import { callPrometheus } from '../../../utils/prometheusUtils';
+
+module.exports = async (fastify: KubeFastifyInstance) => {
+  /**
+   * Pass through API for getting information out of prometheus.
+   * Acts on the user who made the call -- does not need route security; k8s provides that.
+   */
+  fastify.post(
+    '/',
+    async (
+      request: OauthFastifyRequest<{ Body: { query: string; namespace: string } }>,
+    ): Promise<{ code: number; response: PrometheusResponse }> => {
+      const { query, namespace } = request.body;
+
+      return callPrometheus(fastify, request, query, namespace);
+    },
+  );
+};
diff --git a/backend/src/routes/api/status/statusUtils.ts b/backend/src/routes/api/status/statusUtils.ts
@@ -9,7 +9,9 @@ export const status = async (
   request: FastifyRequest,
 ): Promise<{ kube: KubeStatus }> => {
   const kubeContext = fastify.kube.currentContext;
-  const { currentContext, namespace, currentUser, clusterID, clusterBranding } = fastify.kube;
+  const { config, currentContext, namespace, currentUser, clusterID, clusterBranding } =
+    fastify.kube;
+  const { server } = config.getCurrentCluster();
 
   const userName = await getUserName(fastify, request);
   const isAdmin = await isUserAdmin(fastify, userName, namespace);
@@ -33,6 +35,7 @@ export const status = async (
         clusterBranding,
         isAdmin,
         isAllowed,
+        serverURL: server,
       },
     };
   }

diff --git a/backend/src/routes/api/validate-isv/validateISV.ts b/backend/src/routes/api/validate-isv/validateISV.ts
@@ -58,7 +58,13 @@ export const createAccessSecret = async (
   const name = enable.validationSecret;
   const secret = {
     apiVersion: 'v1',
-    metadata: { name, namespace },
+    metadata: {
+      name,
+      namespace,
+      labels: {
+        'opendatahub.io/dashboard': 'true',
+      },
+    },
     type: 'Opaque',
     stringData,
   };
@@ -151,6 +157,9 @@ export const runValidation = async (
       annotations: {
         'cronjob.kubernetes.io/instantiate': 'manual',
       },
+      labels: {
+        'opendatahub.io/dashboard': 'true',
+      },
     },
     spec: cronJob.spec.jobTemplate.spec,
   };