Make get_current_context_identifier() understand EGL.
If we're using EGL and not GLX (typically because we're using GLES,
but also increasingly with desktop GL), we'd always return NULL.
This could FBOs to be confused between contexts.
The intended use case is to have Y'CbCr for encoding output but keep
RGBA around for easier preview. This causes a few effects to need to
send arrays around; it's a bit ugly to special-case them like this,
but I'm concerned about going generic wrt. how good various shader
compilers are to optimize if we went full multi-model everywhere
(without having tested, though).
In practice, we haven't _actually_ supported this since we used integers
in ResampleEffect (and ResampleEffect is a pretty central effect),
so let's be honest with ourselves. (Also, we will soon start using arrays
in some cases, which are cumbersome pre-1.30.) I don't know of any drivers
that support all the other stuff we want but not GLSL 1.30 anyway;
it came with OpenGL 3.0, in 2008.
This actually isn't an ABI break, at least not on the C++ level.
In ycbcr_conversion_effect_test, use a non-float framebuffer.
This way, we let the card convert float-to-int, which we have reasonable
control over, as opposed to glReadPixels(), which is rather unpredictable.
Fixes unit test failures on Broadwell on Linux (Mesa 10.1).
In ResampleEffect, precompute the Lanczos function into a table.
A 2048-element table (with linear interpolation between the elements)
is seemingly enough to get down to beyond float epsilon, and this
saves a lot of CPU time when computing large filter kernels.
Fix a bug where combined fp16 weights would be horribly wrong.
Seemingly weights were always returned as float, and then cast
to fp16_int_t -- without proper conversion! And sum_sq_error
would be calculated based on the correct value, not the broken-
casted one.
It's a small miracle the unit tests didn't catch this; they didn't
until I started introducing small errors for another reason.
Most real-world testing seems to have hit fp32, and thus this
wasn't caught there either.
Also make fp16_int_t a struct so that it is not implicitly
convertible to/from numeric types, so this never ever can happen again.
In ResampleEffect, be more aggressive about giving up on saving bilinear samples.
It turns out that for some kinds of loads, we can't use bilinearity at all
to our benefit, so we spend almost all of our time trying to go through
each line to see how much we can save. Simply send in the minimum number
so far when doing this evaluation to begin with, which means we'll effectively
short-circuit the entire thing pretty fast once we find a line that can save
nothing.
For me, this was needed when I wanted to render directly into
VA-API's encoder buffers, which are always top-left origin (and FBOs
are always bottom-left origin).
Add support for Y'CbCr output split between multiple textures.
This is useful primarily for avoiding copies in later stages;
e.g., when rendering directly into a video encoder buffer.
We support both full planar and NV12-style interleaved Cb+Cr.
You still have to subsample chroma yourself, though; we don't
really support chains that diverge except in the final output node
(and changing resolution would definitely need a bounce;
and even worse, one in a non-fp16 intermediate format).
One would think something as mundane as setting a few uniforms wouldn't
really mean much for performance, but seemingly this is not always so --
I had a real-world shader that counted no less than 55 uniforms.
Of course, not all of these were actually used, but we still have to go
through looking up the name etc. for every single one, every single frame.
Thus, we introduce a new way of dealing with uniforms: Register them before
finalization time, and then EffectChain can store their numbers once and
for all, instead of this repeated lookup. The system is also set up such
that we can go to uniform buffer objects (UBOs) in the very near future.
It's a bit unfortunate that uniform declaration now is removed from the
.frag files, where it sat very nicely, but the alternative would be to
try to parse GLSL, which I'm a bit wary at right now. All effects are
converted, leaving the set_uniform_* functions without any users, but
they are kept around for now in case external effects want them.
This seems to bring 1–2% speedup for my use case; hopefully UBOs will
bring a tiny bit more.
Prepare for better understanding of 10- and 12-bit Y'CbCr.
Seemingly there is trickiness in how to interpret the integer
values that is different from what you'll typically see in R'G'B'
(or just GPUs and TV standards differ on that point as well).
Add an explanatory comment, and add a data member to YCbCrFormat
to prepare for correct 10/12-bit level handlings. We'll stay 8-bit
only for now, though, to avoid an API break for existing clients
for no good reason (there's no 10-bit input, really).
Minor optimization in ResampleEffect: Set less GL state.
In particular, if we can avoid it, use glTexSubImage2D instead of glTexImage2D.
This actually has a real effect, at least on Intel/Linux, where the drive seems
to stall on some mappings.
Of course, this only really helps for things like pans, not zooms.
Note that this is an API break; PaddingEffect now does something else
from what it used to do before when it comes to fractional offsets.
But I feel this is more useful; it allows PaddingEffect to be used
more efficiently for moving things smoothly around.
Also add a concept of border offset which moves the border around
without changing the pixels; useful if you want the subpixel placement
to be done by ResampleEffect (put the integral offset into top/left
and then move the border by the fractional amount it missed).
The assumption is broken whenever a non-integral top or left parameter
is specified. Instead, make an IntegralResampleEffect that enforces
these parameters to be integers, and then mark it as one-to-one sampling.
Collapse passes more aggressively in the face of size changes.
The motivating chain for this change was a case where we had
a SinglePassResampleEffect (the second half of a ResampleEffect)
feeding into a PaddingEffect, feeding into an OverlayEffect.
Currently, since the two former change output size, we'd bounce
to a temporary texture twice (output size changes would always
cause bounces).
However, this is needlessly conservative. The reason for bouncing
when changing output size is really if you want to get rid of
data by downscaling and then later upsampling, e.g. for a blur.
(It could also be useful for cropping, but we don't really use
that right now; PaddingEffect, which does crop, explicitly checks
the borders anyway to set the border color manually.) But in this case,
we are not downscaling at all, so we could just drop the bounce,
saving tons of texture bandwidth.
Thus, we add yet more parameters that effects can specify; first,
that an effect uses _one-to-one_ sampling; that is, that it
will only use its input as-is without sampling
between texels or outside the border (so the different
interpolation and border behavior will be irrelevant).
(Actually, almost all of our effects fall into this category.)
Second, a flag saying that even if an effect changes size,
it doesn't use virtual sizes (otherwise even a one-to-one effect
would de-facto be sampling between texels). If these flags
are set on the input and the output respectively, we can avoid
the bounce, at least unless there's an effect that's _not_
one-to-one further up the chain.
For my motivating case, this folded eight phases into four,
changing ~16.0 ms into ~10.6 ms rendering time. Seemingly
memory bandwidth is a really precious resource on my laptop's
GPU.
This is mostly theoretical; I've never been able to measure any
sort of real change from this. But according to popular cargo-culting,
it might have an effect since there are fewer edge pixels to shade.
Propagate size correctly across effects that change output size.
When propagating size information between effects in a phase,
we'd forget to check if the effect wanted to change size
and use that information instead of our own heuristics.
Fix that.
This is currently a no-op, since right now we always break a phase
when an effect changes output size, but there are very real situations
where we'd be fine with not doing so, so this patch paves the way
for that.
This is useful for debugging slow chains; it can give information
about which phase takes the most time. Right now there seems to be
~5 ms in one of my test chains that disappear into nothing
(ie. show up in the fps counter with vsync off, but not in any
phase), but hopefully we can eventually solve that discrepancy.
Use std::scientific when outputting floats, so we do not get issues with 0.0 being outputs as 0 (which is an int, which cannot always be implicitly converted to float in GLSL).
Really only FlatInput can easily support mipmaps; for things like YCbCrInput
that combine multiple inputs, it's hard (probably not downright impossible,
but at least not immediately obvious without thinking about it a bit) and for
FFTInput it makes no sense.
Thus, we allow an input to say that it can't do this, and then bounce it
to a texture if needed. Hopefully this should happen quite rarely.
This is long overdue, of course; I knew this function was a quick hack,
but didn't realize it was a problem until Christophe Thommeret reported
an issue that looked a lot like this.
Trying to use sprintf and floats right in a portable manner is seemingly
impossible (MinGW doesn't support the per-thread locale stuff), so simply
do it a different way; stop sprintf-ing floats and use std::stringstream
instead. I dislike the iostream interface a lot, but it can do per-stream
locales, which is exactly what we want here.
Dan Dennedy [Thu, 5 Mar 2015 07:41:39 +0000 (23:41 -0800)]
Fix build on OS X and MinGW.
OS X requires the xlocale.h header to define locale_t:
https://developer.apple.com/library/mac/documentation/Darwin/Reference/ManPages/man3/newlocale.3.html
MinGW does not include implementations for newlocale() and uselocale().
Instead, use the previous approach using setlocale().
setlocale() affects the whole process, not just the current thread
as I assumed; uselocale() (available since glibc 2.3, so basically
forever) is per-thread, and also conveniently seems to avoid the
issue of the returned pointer being destroyed (unless the driver
uses the return value of uselocale() as a base, which I really hope
it doesn't).
I'm slightly worried that since this overrides setlocale(), buggy drivers
might get confused when they try to do setlocale() and something else
overrides that precedence, but hopefully this shouldn't be a case.
Also add a unit test for locale handling while we're at it. It doesn't
test multi-threaded behavior, though, only the simple case.
For most users, this is mostly theoretical, as it requires compiling
with -march=native or similar. And these are definitely meant for
vectorizing, although it's still 2-3x as fast to use them as our own
software fallback.
These are supported starting from Haswell, and also by some AMD CPUs.
For the case where the resampling changed every frame (e.g. a zoom),
it just consumed too much CPU to be worth it, especially in memory
management; this is painful because it was an elegant solution to
a tricky problem, but it just has to go for now.
Also drop out to fp32 at the first sight of too-high error.
In ResampleEffect, optimize the bilinear weights on a global scale.
In addition to the individual weight optimization we do when combining samples,
this technique optimizes the weights as a whole, through some linear algebra.
This means it can take into account effects such as multiple bilinear samples
influencing the same coefficient (which normally should not happen, but might
nevertheless due to imprecisions in the stored texture coordinates), or
non-combined sample positions that can't hit the exact middle of the texel.
In practical tests, this is extremely effective; it often reduces the computed
sum of squared coefficient errors by as much as a factor 1000, although I
haven't verified how often it actually saves us from having to do fp32 fallback
with the rather tight error bounds that are in place.
When combining samples, take fp16 rounding into account.
This makes us somewhat more conservative in combining samples;
when we are near the lower/right edges of the image, we are starting
to get close to 1.0, and fp16 just doesn't have enough precision
to give us the 6 or 8 bits of subpixel precision we want (it is
hardly enough to address individual pixels!). In particular, this
can affect zooming with ResampleEffect, as reported by Christophe
Thommeret.
This does not fix all cases (especially not non-power-of-two cases);
for that, we will probably need to be able to fall back to fp32
when we detect fp16 doesn't work well.
We read about twice as many as we should have; the others were
probably just set to 0.0, which has no effect but still burns
arithmetic, unless your driver happens to optimize very aggressively
for this (which I don't think anyone does anymore).
Properly restore the LC_NUMERIC locale after finalizing.
There were two issues here:
1. setlocale(LC_NUMERIC, "C") always returns C, not the previous
locale.
2. The return value of setlocale() may point into static storage,
which may be corrupted when we call into libGL, if e.g.
the shader compiler calls setlocale() on its own.
1. If you're missing some functionality, Movit will now tell you
on stderr what you're missing. (We might suppress this later
if it turns out that people want to init_movit() but are actually
fine with it failing.)
2. Use a table instead of repeated if-then logic, since this started
to become a bit messy after we added OpenGL-version-equivalence
checks.
projects / movit / log