dimanche 8 octobre 2023

OnnxRuntime: Mapping predicted labels to scores

I am using ML.NET for a multiclassification task.

My Model Output is defined as:

        public class ModelOutput
        {
            [ColumnName(@"PredictedLabel")]
            public UInt32 PredictedLabel { get; set; }

            [ColumnName(@"Score")]
            public float[] Score { get; set; }

        }

I am consuming this from a C++ (C++ 11) program using the OnnxRuntime.

The labels are 32-bit unsigned integers, but from a small finite set. I would like to return the top-k (say k=3) labels and their associated scores. Currently, the model returns the scores for all labels, but I do not know the correspondence between the label and the index for that label into the Score array. Is there a way to get that from the OnnxRuntime (in the calling C++ program) from the ONNX model?

For example, this article https://blog.hompus.nl/2020/09/14/get-all-prediction-scores-from-your-ml-net-model/ shows how to get the slot names. The relevant code block is replicated below for quick reference:

var labelBuffer = new VBuffer<ReadOnlyMemory<char>>();
predictionEngine.OutputSchema["Score"]
    .Annotations
    .GetValue("SlotNames", ref labelBuffer);

var labels = labelBuffer.DenseValues().Select(l => l.ToString()).ToArray();

var index = Array.IndexOf(labels, modelOutput.PredictedLabel);
var score = modelOutput.Score[index];

var top10scores = this.labels.ToDictionary(
      l => l,
      l => (decimal)modelOutput.Score[Array.IndexOf(this.labels, l)]
    )
    .OrderByDescending(kv => kv.Value)
    .Take(10);

That would work if my consuming program was also a C# program. In my case, I need to access the prediction through the OnnxRuntime only. I could define my output model to carry the slot names for each prediction and then access it in the C++ program. But that sounds inefficient since the slot names have to be passed around for each prediction. Is there any way around this?

Aucun commentaire:

Enregistrer un commentaire