'AsyncEnumerableReader' reached the configured maximum size, but not using AsyncEnumerable

All Ef Core IQueryable<T> implementations (DbSet<T>, EntityQueryable<T>) also implement the standard IAsyncEnumerable<T> interface (when used from .NET Core 3), so AsEnumerable(), AsQueryable() and AsAsyncEnumerable() simply return the same instance cast to the corresponding interface.

You can easily verify that with the following snippet:

var queryable = _carsDataModelContext.Cars.AsQueryable();
var enumerable = queryable.AsEnumerable();
var asyncEnumerable = queryable.AsAsyncEnumerable();
Debug.Assert(queryable == enumerable && queryable == asyncEnumerable);

So even though you are not returning explicitly IAsyncEnumerable<T>, the underlying object implements it and can be queried for. Knowing that Asp.Net Core is naturally async framework, we can safely assume that it checks if the object implements the new standard IAsyncEnumerable<T>, and uses that behind the scenes instead of IEnumerable<T>.

Of course when you use ToList(), the returned List<T> class does not implement IAsyncEnumerable<T>, hence the only option is to use IEnumerable<T>.

This should explain the 3.1 behavior. Note that before 3.0 there was no standard IAsyncEnumerable<T> interface. EF Core was implementing and returning its own async interface, but the .Net Core infrastructure was unaware of it, thus was unable to use it on behalf of you.

The only way to force the previous behavior without using ToList() / ToArray() and similar is to hide the underlying source (hence the IAsyncEnumerable<T>).

For IEnumerable<T> it's quite easy. All you need is to create custom extension method which uses C# iterator, e.g:

public static partial class Extensions
{
    public static IEnumerable<T> ToEnumerable<T>(this IEnumerable<T> source)
    {
        foreach (var item in source)
            yield return item;
    }
}

and then use

return Ok(_carsDataModelContext.Cars.ToEnumerable());

If you want to return IQueryable<T>, the things get harder. Creating custom IQueryable<T> wrapper is not enough, you have to create custom IQueryProvider wrapper to make sure composing over returned wrapped IQueryable<T> would continue returning wrappers until the final IEnumerator<T> (or IEnumerator) is requested, and the returned underlying async enumerable is hidden with the aforementioned method.

Here is a simplified implementation of the above:

public static partial class Extensions
{
    public static IQueryable<T> ToQueryable<T>(this IQueryable<T> source)
        => new Queryable<T>(new QueryProvider(source.Provider), source.Expression);

    class Queryable<T> : IQueryable<T>
    {
        internal Queryable(IQueryProvider provider, Expression expression)
        {
            Provider = provider;
            Expression = expression;
        }
        public Type ElementType => typeof(T);
        public Expression Expression { get; }
        public IQueryProvider Provider { get; }
        public IEnumerator<T> GetEnumerator() => Provider.Execute<IEnumerable<T>>(Expression)
            .ToEnumerable().GetEnumerator();
        IEnumerator IEnumerable.GetEnumerator() => GetEnumerator();
    }

    class QueryProvider : IQueryProvider
    {
        private readonly IQueryProvider source;
        internal QueryProvider(IQueryProvider source) => this.source = source;
        public IQueryable CreateQuery(Expression expression)
        {
            var query = source.CreateQuery(expression);
            return (IQueryable)Activator.CreateInstance(
                typeof(Queryable<>).MakeGenericType(query.ElementType),
                this, query.Expression);
        }
        public IQueryable<TElement> CreateQuery<TElement>(Expression expression)
            => new Queryable<TElement>(this, expression);
        public object Execute(Expression expression) => source.Execute(expression);
        public TResult Execute<TResult>(Expression expression) => source.Execute<TResult>(expression);
    }
}

The query provider implementation is not fully correct, because it assumes that only the custom Queryable<T> will call Execute methods for creating IEnumerable<T>, and external calls will be used only for immediate methods like Count, FirstOrDefault, Max etc., but it should work for this scenario.

Other drawback of this implementation is that all EF Core specific Queryable extensions won't work, which might be an issue/showstopper if OData $expand relies on methods like Include / ThenInclude. But fixing that requires more complex implementation digging into EF Core internals.

With that being said, the usage of course would be:

return Ok(_carsDataModelContext.Cars.ToQueryable());