First, the Variance category has four static inner classes

private static class EqualitySetDeleteFilter<T> extends Filter<T>
private static class PositionSetDeleteFilter<T> extends Filter<T>
private static class PositionStreamDeleteFilter<T> extends CloseableGroup implements CloseableIterable<T>
private static class DataFileFilter<T extends StructLike> extends Filter<T>
Copy the code

EqualitySetDeleteFilter, PositionSetDeleteFilter, DataFileFilter all inherit from Filter abstract class. We need to implement a protected abstract Boolean shouldKeep(T item); Method to determine whether the record is retained, the following is an implementation of EqualitySetDeleteFilter

  private static class EqualitySetDeleteFilter<T> extends Filter<T> {
    private final StructLikeSet deletes;
    private final Function<T, StructLike> extractEqStruct;

    protected EqualitySetDeleteFilter(Function
       
         extractEq, StructLikeSet deletes)
       ,> {
      this.extractEqStruct = extractEq;
      this.deletes = deletes;
    }

    @Override
    protected boolean shouldKeep(T row) {
      return !deletes.contains(extractEqStruct.apply(row));
    }
  }
Copy the code

The member variables are private final StructLikeSet deletes; Private final Function

extractEqStruct; Is a method for extracting equal struct types, the parameter type of the method is T, the return value type is StructLike, the method for translating data into primary keys, keys that are expected to have a logic of Variance and do not include row are retained in the shouldKeep method.
,>

StructLikeSet class is a class that implements Set, the main member variables

public static StructLikeSet create(Types.StructType type) {
    return new StructLikeSet(type);
  }

  private final Types.StructType type;
  private final Set<StructLikeWrapper> wrapperSet;
  private final ThreadLocal<StructLikeWrapper> wrappers;

  private StructLikeSet(Types.StructType type) {
    this.type = type;
    this.wrapperSet = Sets.newHashSet();
    this.wrappers = ThreadLocal.withInitial(() -> StructLikeWrapper.forType(type));
  }
Copy the code

StructType = StructLike; private final Set

wrapperSet; For a collection of real stored data,