That's kinda neat. Do you do any decoding tricks to handle varying processing capabilities across device models and ipads? Like shortening ctx length or maybe looking into the LFM models that have MoEs with like 1b active?
That's kinda neat. Do you do any decoding tricks to handle varying processing capabilities across device models and ipads? Like shortening ctx length or maybe looking into the LFM models that have MoEs with like 1b active?
