A filter is introduced

At present, there are many filters on the market, but there are only a few overall categories, which are processed in fragment Shader. At present, the most commonly used filter is luT filter and adjust the RGB curve filter. Other type changes are much the same.

Construction of dynamic filters

To implement the dynamic download of filters, we next implement a set of json parameters for the filter, Parameters include filter type, filter name, Vertex shader, Fragment shader file, uniform variable list, texture image bound to uniform variable, default filter strength, whether to have texture width and height offset, music path, whether music is played in a loop and so on. Json and its fields are described as follows:

{"filterList": [{"type": "filter", // indicates the filter type. Currently, filter is a normal filter, and other types of filters will be added later. "", // vertex shader file name "fragmentShader": GLSL ", // uniformList":["blowoutTexture", "overlayTexture", "mapTexture"], // uniformData: {blowoutTexture: "blowout.png", "overlayTexture": "Overlay. PNG ", "mapTexture": "map.png"}, "strength": 1.0, // Default filter strength between 0.0 and 1.0 "texelOffset": 0, // Whether to support the width and height offset value, that is, to pass 1.0f/width, 1.0f/height to shader "audioPath": "", // music path "audioLooping": 1 // whether to loop music}]}Copy the code

Now that we have json, we need to decode the filter parameter object as follows:

/** * decodeFilterData(String folderPath) throws public static DynamicColor decodeFilterData(String folderPath) IOException, JSONException { File file = new File(folderPath, "json"); String filterJson = FileUtils.convertToString(new FileInputStream(file)); JSONObject jsonObject = new JSONObject(filterJson); DynamicColor dynamicColor = new DynamicColor(); dynamicColor.unzipPath = folderPath; if (dynamicColor.filterList == null) { dynamicColor.filterList = new ArrayList<>(); } JSONArray filterList = jsonObject.getJSONArray("filterList"); for (int filterIndex = 0; filterIndex < filterList.length(); filterIndex++) { DynamicColorData filterData = new DynamicColorData(); JSONObject jsonData = filterList.getJSONObject(filterIndex); String type = jsonData.getString("type"); If ("filter".equals(type)) {filterdata.name = jsondata.getString ("name"); filterData.vertexShader = jsonData.getString("vertexShader"); filterData.fragmentShader = jsonData.getString("fragmentShader"); JSONArray uniformList = jsonData.getJSONArray("uniformList"); for (int uniformIndex = 0; uniformIndex < uniformList.length(); uniformIndex++) { String uniform = uniformList.getString(uniformIndex); filterData.uniformList.add(uniform); JSONObject uniformData = jsondata. getJSONObject("uniformData"); if (uniformData ! = null) { Iterator<String> dataIterator = uniformData.keys(); while (dataIterator.hasNext()) { String key = dataIterator.next(); String value = uniformData.getString(key); filterData.uniformDataList.add(new DynamicColorData.UniformData(key, value)); } } filterData.strength = (float) jsonData.getDouble("strength"); filterData.texelOffset = (jsonData.getInt("texelOffset") == 1); filterData.audioPath = jsonData.getString("audioPath"); filterData.audioLooping = (jsonData.getInt("audioLooping") == 1); } dynamicColor.filterList.add(filterData); } return dynamicColor; }Copy the code

Implementation of filters

After decoding the filter parameters, we then implement the dynamic filter rendering process. To facilitate the filter construction, we create a filter resource loader with the following code:

/** * public class DynamicColorLoader {private static final String TAG = "DynamicColorLoader"; // Filter folder private String mFolderPath; Private DynamicColorData mColorData; Private ResourceDataCodec mResourceCodec; // Private final WeakReference<DynamicColorBaseFilter> mWeakFilter; Private HashMap<String, Integer> mUniformHandleList = new HashMap<>(); Private int[] mTextureList; Private int mTexelWidthOffsetHandle = opengLutils.gl_not_init; // Private int mTexelWidthOffsetHandle = opengLutils.gl_not_init; private int mTexelHeightOffsetHandle = OpenGLUtils.GL_NOT_INIT; private int mStrengthHandle = OpenGLUtils.GL_NOT_INIT; Private float mStrength = 1.0f; Private float mTexelWidthOffset = 1.0f; Private float mTexelHeightOffset = 1.0f; public DynamicColorLoader(DynamicColorBaseFilter filter, DynamicColorData colorData, String folderPath) { mWeakFilter = new WeakReference<>(filter); mFolderPath = folderPath.startsWith("file://") ? folderPath.substring("file://".length()) : folderPath; mColorData = colorData; mStrength = (colorData == null) ? F: 1.0 colorData. Strength; Pair pair = ResourceCodec.getResourceFile(mFolderPath); if (pair ! = null) { mResourceCodec = new ResourceDataCodec(mFolderPath + "/" + (String) pair.first, mFolderPath + "/" + pair.second); } if (mResourceCodec ! = null) { try { mResourceCodec.init(); } catch (IOException e) { Log.e(TAG, "DynamicColorLoader: ", e); mResourceCodec = null; } } if (! TextUtils.isEmpty(mColorData.audioPath)) { if (mWeakFilter.get() ! = null) { mWeakFilter.get().setAudioPath(Uri.parse(mFolderPath + "/" + mColorData.audioPath)); mWeakFilter.get().setLooping(mColorData.audioLooping); } } loadColorTexture(); } / texture * * * * load/private void loadColorTexture () {if (mColorData. UniformDataList = = null | | mColorData.uniformDataList.size() <= 0) { return; } mTextureList = new int[mColorData.uniformDataList.size()]; for (int dataIndex = 0; dataIndex < mColorData.uniformDataList.size(); dataIndex++) { Bitmap bitmap = null; if (mResourceCodec ! = null) { bitmap = mResourceCodec.loadBitmap(mColorData.uniformDataList.get(dataIndex).value); } if (bitmap == null) { bitmap = BitmapUtils.getBitmapFromFile(mFolderPath + "/" + String.format(mColorData.uniformDataList.get(dataIndex).value)); } if (bitmap ! = null) { mTextureList[dataIndex] = OpenGLUtils.createTexture(bitmap); bitmap.recycle(); } else { mTextureList[dataIndex] = OpenGLUtils.GL_NOT_TEXTURE; Public void onBindUniformHandle(int programHandle) {if (programHandle ==  OpenGLUtils.GL_NOT_INIT || mColorData == null) { return; } mStrengthHandle = GLES30.glGetUniformLocation(programHandle, "strength"); if (mColorData.texelOffset) { mTexelWidthOffsetHandle = GLES30.glGetUniformLocation(programHandle, "texelWidthOffset"); mTexelHeightOffsetHandle = GLES30.glGetUniformLocation(programHandle, "texelHeightOffset"); } else { mTexelWidthOffsetHandle = OpenGLUtils.GL_NOT_INIT; mTexelHeightOffsetHandle = OpenGLUtils.GL_NOT_INIT; } for (int uniformIndex = 0; uniformIndex < mColorData.uniformList.size(); uniformIndex++) { String uniformString = mColorData.uniformList.get(uniformIndex); int handle = GLES30.glGetUniformLocation(programHandle, uniformString); mUniformHandleList.put(uniformString, handle); Public void onInputSizeChange(int width, @param height, @param height) Int height) {mTexelWidthOffset = 1.0f/width; MTexelHeightOffset = 1.0f/height; } / / public void onDrawFrameBegin() {if (mStrengthHandle! = OpenGLUtils.GL_NOT_INIT) { GLES30.glUniform1f(mStrengthHandle, mStrength); } if (mTexelWidthOffsetHandle ! = OpenGLUtils.GL_NOT_INIT) { GLES30.glUniform1f(mTexelWidthOffsetHandle, mTexelWidthOffset); } if (mTexelHeightOffsetHandle ! = OpenGLUtils.GL_NOT_INIT) { GLES30.glUniform1f(mTexelHeightOffsetHandle, mTexelHeightOffset); } if (mTextureList == null || mColorData == null) { return; } // bind textures one by one for (int dataIndex = 0; dataIndex < mColorData.uniformDataList.size(); dataIndex++) { for (int uniformIndex = 0; uniformIndex < mUniformHandleList.size(); UniformIndex++) {// if the uniform variable exists, Integer is directly binding texture handle = mUniformHandleList. Get (mColorData. UniformDataList. Get (dataIndex). Uniform); if (handle ! = null && mTextureList[dataIndex] ! = OpenGLUtils.GL_NOT_TEXTURE) { OpenGLUtils.bindTexture(handle, mTextureList[dataIndex], dataIndex + 1); }}}} /** * release */ public void release() {if (mTextureList! = null && mTextureList.length > 0) { GLES30.glDeleteTextures(mTextureList.length, mTextureList, 0); mTextureList = null; } if (mWeakFilter.get() ! = null) { mWeakFilter.clear(); }} @param strength */ public void setStrength(float strength) {mStrength = strength; }}Copy the code

Then we build a base class of DynamicColorFilter, which is convenient for adding other types of filters. The code is as follows:

Public class DynamicColorBaseFilter extends GLImageAudioFilter {//  protected DynamicColorLoader mDynamicColorLoader; public DynamicColorBaseFilter(Context context, DynamicColorData dynamicColorData, String unzipPath) { super(context, (dynamicColorData == null || TextUtils.isEmpty(dynamicColorData.vertexShader)) ? VERTEX_SHADER : getShaderString(context, unzipPath, dynamicColorData.vertexShader), (dynamicColorData == null || TextUtils.isEmpty(dynamicColorData.fragmentShader)) ? FRAGMENT_SHADER_2D : getShaderString(context, unzipPath, dynamicColorData.fragmentShader)); mDynamicColorData = dynamicColorData; mDynamicColorLoader = new DynamicColorLoader(this, mDynamicColorData, unzipPath); mDynamicColorLoader.onBindUniformHandle(mProgramHandle); } @Override public void onInputSizeChanged(int width, int height) { super.onInputSizeChanged(width, height); if (mDynamicColorLoader ! = null) { mDynamicColorLoader.onInputSizeChange(width, height); } } @Override public void onDrawFrameBegin() { super.onDrawFrameBegin(); if (mDynamicColorLoader ! = null) { mDynamicColorLoader.onDrawFrameBegin(); } } @Override public void release() { super.release(); if (mDynamicColorLoader ! = null) { mDynamicColorLoader.release(); }} @param strength */ public void setStrength(float strength) {if (mDynamicColorLoader! = null) { mDynamicColorLoader.setStrength(strength); @param unzipPath * @param shaderName * @return */ protected static String getShaderString(Context context, String unzipPath, String shaderName) { if (TextUtils.isEmpty(unzipPath) || TextUtils.isEmpty(shaderName)) { throw new IllegalArgumentException("shader is empty!" ); } String path = unzipPath + "/" + shaderName; if (path.startsWith("assets://")) { return OpenGLUtils.getShaderFromAssets(context, path.substring("assets://".length())); } else if (path.startsWith("file://")) { return OpenGLUtils.getShaderFromFile(path.substring("file://".length())); } return OpenGLUtils.getShaderFromFile(path); }}Copy the code

Next we build the dynamic filter group, because dynamic filters can have multiple filters combined. The code is as follows:

public class GLImageDynamicColorFilter extends GLImageGroupFilter { public GLImageDynamicColorFilter(Context context, DynamicColor dynamicColor) { super(context); / / whether the data is the if (dynamicColor = = null | | dynamicColor. FilterList = = null | | TextUtils. IsEmpty (dynamicColor. UnzipPath)) { return; } // Add filter for (int I = 0; i < dynamicColor.filterList.size(); i++) { mFilters.add(new DynamicColorFilter(context, dynamicColor.filterList.get(i), dynamicColor.unzipPath)); }} @param strength @public void setStrength(float strength) {for (int I = 0; i < mFilters.size(); i++) { if (mFilters.get(i) ! = null && mFilters.get(i) instanceof DynamicColorBaseFilter) { ((DynamicColorBaseFilter) mFilters.get(i)).setStrength(strength); }}}}Copy the code

conclusion

The basic dynamic filters are relatively simple to implement, and are simply json parameters, shaders, uniform variables, and texture bindings that need to be built dynamically. The effect is as follows:




Dynamic filter effect


This effect is achieved by decompressing the compressed package resources in the Asset directory. You only need to provide a zip package containing shader, texture resources, and JSON to change the filter.

For detailed implementation process, please refer to my open source project: CainCamera