Feature requests - Output + Implementations

[niscinfo]

Hello to all,

I'm opening this issue, for 2 feature requests.

First one would be to be able to get the output of kubectl-cost formatted in CSV and/or JSON.
From the [https://github.com/kubecost/docs/blob/master/allocation.md#querying] documentation it is possible by API calls ("format").

Second would be the possibility to use other query stuff from querying (same link as above) , like the aggregation options, and the filters.

Thanks for all the work and in advance for further

Regards,

[dwbrown2]

@niscinfo thanks for this request! I think it's cool.

have any example requests that you're picturing for filtering? just want to get a sense for any specific use case you may have.

cc @kaelanspatel @michaelmdresser

[michaelmdresser]

Thank you for the feedback @niscinfo! I'm glad you're using kubectl cost. We're quite busy working on Kubecost right now, so I suspect we won't be able to make serious progress here for a couple of months. If you'd like to take a stab at any of these features before we get to them, we absolutely accept contributions and I would be happy to walk through the code with you or answer any questions you have. You can join our Slack and message me, Michael Dresser.

Output formats

I'm undecided if we should use the format parameter of the API. While it would improve consistency between kubectl cost and the "download CSV" functionality available on the frontend, it might make kubectl cost more fragile in the event of a change to the format behavior of the allocation and/or asset APIs. This is certainly worth discussing!

(Note that the Assets API does have format=csv as an option)

I think what we're finding here is that the ergonomic benefits of kubectl cost handling port-forwarding for you are not insignificant @dwbrown2. I think that's pretty cool 😁

Formatting output as CSV

This should be fairly straightforward even if we don't use the format query parameter. The table builders makeAllocationTable and makeAssetTable create a pretty generic structure using go-pretty which support rendering as CSV! See the TUIs usage of CSV rendering.

We could add a CLI flag like --output-format csv that uses that CSV rendering functionality.

Formatting output as JSON

We could add a similar --output-format json for this functionality and simply write out the raw JSON response from the API instead of converting it to a table. If we do implement this, it would be wise to just use the API's format=csv option for the "Formatting output as CSV" request to maintain consistency of data and formatting.

Additional query options

Filters

As Webb asked, what filters are you specifically looking for that are missing? This could be addressed as part of my ideas in the next section:

Aggregations

I think there's a huge opportunity here to both simplify the code and expand functionality. kubectl cost was written very iteratively and each aggregation is its own file, even if its using the same API. There is a lot of code duplication due to this development approach. If designed correctly, I think we could reduce LOC while adding the ability to perform all types of aggregations. My only concern is that we would lose some UX ergonomics because kubectl cost currently has an opinion about which flags/filters can be passed to each aggregation via the aggregation-specific options (example).

My vision is that this would look something like:

kubectl cost [K8s query flags like --kubeconfig] [aggregation] [Kubecost query flags like --namespace]

Where [aggregation] could be namespace, namespace,controller, cluster,namespace, etc. The Kubecost query flags would have to be manually added, but because this is all querying the same API it won't be a huge problem.

[niscinfo]

Hi @michaelmdresser

Thanks for quick answers.
Just to expose the "why" this request :
We have Kubernetes Clusters multi tenancy and we do re-invoicing in departments of my company.
We can use labels or namespace names to identitfy which departement will be charged.
Sorting in JSON/CSV allows us to easily interact will billing tools. For the moment we do some nasty 'grep', 'awk' ,

Any idea is welcome

Regards,

[michaelmdresser]

Hey @niscinfo,

Sorting in JSON/CSV allows us to easily interact will billing tools. For the moment we do some nasty 'grep', 'awk'

At the end of the day, kubectl cost is just interacting with the Kubecost API via port-forwarding in the code using the Kubernetes client SDK/API. In the long term, I certainly agree with you that updating kubectl cost is more ergonomic and we'd like to get there! To solve your short term pain and get JSON to work with, you could do something like this:

# In one terminal
kubectl port-forward \
    --namespace kubecost \
    service/kubecost-cost-analyzer \
    9090

# In another terminal
curl -G 'localhost:9090/model/allocation' \
    -d 'window=7d' \
    -d 'aggregate=namespace' \
    -d 'accumulate=true'

Reference our Allocation API docs for other parameters!

[niscinfo]

Hey @michaelmdresser

Thank you for the trick !
It will help surely, but of course it's not so easy and automated as
kubectl cost namespace --output-format csv :-)

But it's a enhancement so no hurry about that.

Thanks for all !