UseNewtonsoftSerialization formats other responses as well

I want to use UseNewtonsoftSerialization for my custom Api Controllers.

// Startup.cs
services.UseNewtonsoftSerialization(typeof(BaseJsonApiController), settings =>
{
	settings.Converters.Add(new StringEnumConverter(new DefaultNamingStrategy()));
	settings.ContractResolver = new DefaultContractResolver
	{
		NamingStrategy = new CamelCaseNamingStrategy(true, true, true)
	};
	settings.StringEscapeHandling = StringEscapeHandling.EscapeHtml;
	settings.ReferenceLoopHandling = ReferenceLoopHandling.Ignore;
});

This setting some how also affects loading of form data in backend when viewing form submissions.

/epiui/EPiServer.Forms.UI/Stores/formsdata/?parent=xxx_yyy

If I remove the UseNewtonsoftSerialization the form submissions data starts working correctly again.

Any ideas?