Adding the design doc covering the tensor types #316

tannergooding · 2024-02-26T21:31:15Z

No description provided.

stephentoub · 2024-02-26T22:31:31Z

accepted/2024/tensors/README.md

+    // IEquatable
+
+    bool Equals(TSelf other);
+
+    // IEqualityOperators
+
+    static bool operator ==(TSelf left, TSelf right);
+    static bool operator ==(TSelf left, T right);
+
+    static bool operator !=(TSelf left, TSelf right);
+    static bool operator !=(TSelf left, T right);


How is equality defined? Rank, strides, and all elements are equal?

Yes. It would then fail for something like comparing a 4x1 vs a 1x4 as the first is four rows vs 4 columns. A user would need to reshape to compare them.

It would then fail for something like comparing a 4x1 vs a 1x4

Fail as in throwing an exception? Would that carry over to the IEquatable<T> implementation or the Equals(object) method? Convention for such methods is to return false for comparands whose type or shape doesn't match.

Fail would mean returning false, as is required by IEquatable and typical for equality implementations

stephentoub · 2024-02-26T22:34:04Z

accepted/2024/tensors/README.md

+    static TSelf Empty { get; }
+
+    bool IsEmpty { get; }


How is empty defined? If I have a two-dimensional tensor where the length of both dimensions is 0, I assume that's empty? Do we need to an empty singleton for different numbers of dimensions, or just having whatever shape Empty returns is fine?

How is empty defined?

All lengths are 0, just like for array.

Do we need to an empty singleton for different numbers of dimensions, or just having whatever shape Empty returns is fine?

For empty in particular, I don't think we need one per rank. Rather, it can be a special marker value that is treated as compatible with other sizes, much as scalar is implicitly broadcast to every element for many of the operations.

stephentoub · 2024-02-26T22:36:21Z

accepted/2024/tensors/README.md

+    static TSelf Empty { get; }
+
+    bool IsEmpty { get; }
+    bool IsPinned { get; }


I assume the array-backed instances we create will not be pinned by default?

Correct. We would treat it just like a normal new T[] unless the user explicitly opts for it otherwise.

stephentoub · 2024-02-26T22:40:53Z

accepted/2024/tensors/README.md

+    static implicit operator TensorSpan<T>(TSelf value);
+    static implicit operator TensorReadOnlySpan<T>(TSelf value);


How do I use any of this with e.g. TensorPrimitives APIs that take ReadOnlySpan<T> and Span<T>? I don't see any conversions from Tensor{ReadOnly}Span<T> to {ReadOnly}Span<T>.

Converting from TensorSpan to Span is "unsafe" due to the loss of rank/strides and potential for overflow on total length.

Internally, we'd use the MemoryMarshal.CreateSpan APIs to handle it in appropriately sized chunks.

stephentoub · 2024-02-26T22:43:51Z

accepted/2024/tensors/README.md

+    static implicit operator TensorSpan<T>(TSelf value);
+    static implicit operator TensorReadOnlySpan<T>(TSelf value);


This implies that any ITensor implementation would need to always store its data contiguously? Or would an implementation dynamically change to use a contiguous implementation if it wasn't already when one of these methods was used?

It is currently assumed to be contiguous always yes. A user who wanted it separately should use a Tensor<T>[] instead (functionally like a jagged array). This makes it clearer when there is contiguous vs non-contiguous, better fits how some other libraries expose/handle the concept, and allows the separate allocations to then have their lifetimes managed independently.

stephentoub · 2024-02-26T22:44:39Z

accepted/2024/tensors/README.md

+    ref T GetPinnableReference();
+    TSelf Slice(params ReadOnlySpan<Range<nint>> ranges);
+
+    void Clear();


How is Clear defined? Is it the equivalent of Fill(default), or is it changing something about the rank / strides?

Same as Array, so yes equivalent to Fill(default) but without needing to explicitly check that the fill value is bitwise zero.

stephentoub · 2024-02-26T22:48:26Z

accepted/2024/tensors/README.md

+{
+    // Effectively mirror the TensorPrimitives surface area. Following the general pattern
+    // where we will return a new tensor and an overload that takes in the destination explicitly.
+    // * public static ITensor<T> Abs<T>(ITensor<T> x) where T : INumberBase<T>;


What kind of ITensor<T> to these methods produce? While an implementation detail, presumably it's going to end up being a Tensor<T> (except in corner cases where we might be able to use a singleton)? Would there be any benefit to finding ways to create these the same as the inputs, e.g. if you passed in a FooTensor<T> to Abs, you'd get back a new FooTensor?

It might be desirable to allow creating the same kind of ITensor and is probably worth some more discussion.

However, we aren't actually allowing the user to provide the implementation for Abs here (to help minimize the number of interfaces, etc) and the inputs may not all be the same (consider MultiplyAdd which takes in 3 different tensors). So I think such support would likely require some new kind of ITensorAllocator (or better name) that allows the user to customize how temporaries are created. I think that gets more into the concept of how temporaries work beyond the barebones support and is what we'll want to discuss in our offline meeting.

stephentoub · 2024-02-26T22:49:51Z

accepted/2024/tensors/README.md

+    // Without this support, we end up having to have `TensorNumber<T>`, `TensorBinaryInteger<T>`, etc
+    // as we otherwise cannot correctly expose the operators based on what `T` supports


Is that really the fallback? I'd have expected the fallback would be named extension methods until extension operators are supported, at which point we would add operators that map to the named methods. It'd be unfortunate if we shipped types we knew were going to be immediate legacy.

I didn't explain this well enough. The proposal is meant to go with what you say. Exposing explicit Add methods and let that eventually become extension operator +. If we don't do that, the alternative is to do what generic math did, which while the correct choice for the fundamental numeric interfaces, is much less ideal for the general purpose patterns built on top of that, like Tensor<T>.

stephentoub · 2024-02-26T22:50:49Z

accepted/2024/tensors/README.md

+
+    // APIs that would return `bool` like `GreaterThan` are split into 3. Following the general
+    // pattern already established for our SIMD vector types.
+    // * public static ITensor<T> GreaterThan(ITensor<T> x, ITensor<T> y);


This returns ITensor<T> rather than ITensor<bool>? I understand why we do Vector<T> for the vector types, but that doesn't seem like the right answer here.

Typo. This is meant to be ITensor<bool> here.

Notably this is one of the places where what's best depends on what T is. If its a primitive type and going to be vectorized, bool is a terrible option. If its a user-defined type, then its one of the better choices. Some people may even want a bitmask result instead.

I want us to discuss this a little bit and see if we also want to expose any special APIs for the case of the user wanting ITensor<T> or BitMask returned or if we want to hold off on that until there's a need. -- Functionally we'll end up implementing the former and then narrowing to byte before storage for the vectorized case regardless, so exposing it is just a question of naming the API we'll already be writing.

stephentoub · 2024-02-26T22:55:27Z

accepted/2024/tensors/README.md

+    public static Tensor<T> Create(bool mustPin, ReadOnlySpan<nint> lengths);
+    public static Tensor<T> Create(bool mustPin, ReadOnlySpan<nint> lengths, ReadOnlySpan<nint> strides);
+
+    public static Tensor<T> Create(T* address, ReadOnlySpan<nint> lengths);
+    public static Tensor<T> Create(T* address, ReadOnlySpan<nint> lengths, ReadOnlySpan<nint> strides);


How would we expect existing tensor types to integrate? e.g. TorchSharp's Tensor<T>, would the idea be that it would implement ITensor<Tensor<T>, T>, or otherwise expose such an implementation?

Yes. Those tensor types would choose one or more of:

Implement ITensor<TSelf, T>

Provide conversions to/from TensorSpan<T>

Provide access to the underlying T* so that users can unsafely create a Tensor<T> or TensorSpan<T> over it

What they get out of it varies between the options, but it gives flexibility for how integrated they want to be with whatever the BCL provides here. The minimal interface should make it very easy for opt-in, even if explicit implementation only.

eiriktsarpalis · 2024-02-27T14:57:40Z

accepted/2024/tensors/README.md

+    ref T this[params ReadOnlySpan<nint> indices] { get; }
+
+    static implicit operator TensorSpan<T>(TSelf value);
+    static implicit operator TensorReadOnlySpan<T>(TSelf value);


Both ReadOnlyTensorSpan<T> and TensorReadOnlySpan<T> names are being used in this doc.

eiriktsarpalis · 2024-02-27T15:12:50Z

accepted/2024/tensors/README.md

+
+    bool IsEmpty { get; }
+    bool IsPinned { get; }
+    int Rank { get; }


In the context of linear algebra, the term "rank" denotes the number of linearly independent rows contained within the tensor (i.e. it is a number derived from the contents of the tensor rather than its shape). Do we prefer to follow .NET MD array terminology here or use something comparable to what other tensor libraries are doing (e.g. ndim, TotalDimensions, etc)

tannergooding added 2 commits February 23, 2024 12:46

Adding the design doc covering the tensor types

70af54b

Ensure ROTensorSpan.CopyTo takes a mutable span

2f16026

stephentoub reviewed Feb 26, 2024

View reviewed changes

eiriktsarpalis reviewed Feb 27, 2024

View reviewed changes

This was referenced Apr 11, 2024

Add a BCL tensor API to represent multi-dimensional data for machine learning dotnet/runtime#24385

Closed

Tensor<T> and supporting types dotnet/runtime#100924

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Adding the design doc covering the tensor types #316

Adding the design doc covering the tensor types #316

tannergooding commented Feb 26, 2024

stephentoub Feb 26, 2024

tannergooding Feb 27, 2024

eiriktsarpalis Feb 27, 2024

tannergooding Feb 27, 2024

stephentoub Feb 26, 2024

tannergooding Feb 27, 2024

stephentoub Feb 26, 2024

tannergooding Feb 27, 2024

stephentoub Feb 26, 2024

tannergooding Feb 27, 2024

stephentoub Feb 26, 2024

tannergooding Feb 27, 2024

stephentoub Feb 26, 2024

tannergooding Feb 27, 2024

stephentoub Feb 26, 2024

tannergooding Feb 27, 2024

stephentoub Feb 26, 2024

tannergooding Feb 27, 2024

stephentoub Feb 26, 2024

tannergooding Feb 27, 2024

stephentoub Feb 26, 2024

tannergooding Feb 27, 2024

eiriktsarpalis Feb 27, 2024

eiriktsarpalis Feb 27, 2024

		static implicit operator TensorSpan<T>(TSelf value);
		static implicit operator TensorReadOnlySpan<T>(TSelf value);

		// Without this support, we end up having to have `TensorNumber<T>`, `TensorBinaryInteger<T>`, etc
		// as we otherwise cannot correctly expose the operators based on what `T` supports

Adding the design doc covering the tensor types #316

Are you sure you want to change the base?

Adding the design doc covering the tensor types #316

Conversation

tannergooding commented Feb 26, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment