Preface:
In response to the epidemic prevention requirements recently, the development of a set of face recognition + wrist temperature measurement + ID card + health code access gate project, face recognition is the rainbow soft face recognition algorithm. Face recognition + temperature measurement, brush ID card + temperature measurement, brush health code + temperature measurement for the door conditions. (Attached source code at the end of the article)
Hardware and software Environment
The platform is Android platform, using Kotlin + Java mixed rainbow software SDK version is the latest 4.0 can wear masks to identify terminal camera binocular camera module IR living recognition scanner wharf, temperature measurement head, ID card reader are all equipment of our company, will not be introduced
UI and machine display
Directions for use
Face recognition through automatic temperature measurement, and then upload temperature and personnel information to the background, background determine whether the temperature is abnormal, and save personnel traffic records
Overall project process
Face registration: face registration using another terminal and small program registration two ways, here only said small program. The user uses the small program to collect face photos to upload to the server -> face terminal from the service regular request terminal has not registered face to the server -> terminal get face photos after the registration to the local. In addition to the periodic request to delete and change the face information, and then do local delete change operation. (Directly synching face photos instead of eigenvalues is because Hongsoft currently does not have a face recognition SDK for small programs)
Open the door to face recognition + temperature measurement, brush ID card + temperature measurement, brush health code + temperature measurement for open the door conditions. This article mainly explains the face + temperature measurement
Introduction to the main categories of the project
PullDataServerHelper pulls the help class of face information to register, delete, and change the information after receiving the information
DataSyncService Indicates the data synchronization service. This type is the server. It periodically invokes PullDataServerHelper to make network requests
Facedb package This package is the database operation related files, the project data operation using Greendao, if you do not know about it, it is very easy to use.
Some of the project on the first say so much, the article will finally attach the source code, then focus on the use of some Rainbow soft SDK
Face recognition part (core code)
1. The activation of the SDK
SDK is permanently activated once, and cannot be activated multiple times. In this paper, online activation is used. The binding activation code of the terminal is recorded in the back end, and the APP requests the activation code of the back end with the unique identifier of the terminal. Check whether it has been activated before activation, and continue the activation operation only if it has not been activated. The following code is shown:
fun Active() {/ / for activation files val activeFileInfo = activeFileInfo (val). Code = FaceEngine getActiveFileInfo (mContext activeFileInfo)if(code == errorInfo.mok) {// IsActive.value = has been activatedtrue
return
} elseVar sdkKey = readString(mContext, Constants.APP_SDK_KEY) var appId = readString(mContext, Constants. Constants.APP_ID_KEY ) var activeKey = readString( mContext, Constants.APP_ACTIVE_KEY )if(sdkkey.isNullorempty ()) {// getSdkInfo()}else {
val code1 = FaceEngine.activeOnline(
mContext,
activeKey,
appId,
sdkKey
)
if (code1 == ErrorInfo.MOK) {
isActive.value = true
return
} else {
getSdkInfo()
}
}
}
}
private fun getSdkInfo() {
RetrofitManager.getInstance().createReq(ApiServer::class.java)
.getSdkInfo(AppUtils.getMac())
.subscribeOn(Schedulers.io())
.observeOn(AndroidSchedulers.mainThread())
.subscribe(object : BaseObserver<SdkInfoResult>() {
override fun onSuccees(data: SdkInfoResult) {
if(data.code == 200 && null ! = data.data) { write(mContext, Constants.APP_SDK_KEY, data.data.SdkKey) write(mContext, Constants.APP_ID_KEY, data.data.AppId) write(mContext, Constants.APP_ACTIVE_KEY, data.data.ActiveKey) val code1 = FaceEngine.activeOnline( mContext, data.data.activeKey, data.data.appId, data.data.sdkKey )if (code1 == ErrorInfo.MOK) {
isActive.value = true
return
} else {
isActive.value = false
}
}
}
override fun onFailure(message: String?) {
isActive.value = false}})}Copy the codeThe various attributes of initialization are explained in detail in the official documentation, which will not be described here
public void init() {
Context context = CustomApplication.Companion.getMContext();
FaceServer.getInstance().init(context);
ftEngine = new FaceEngine();
int ftEngineMask = FaceEngine.ASF_FACE_DETECT | FaceEngine.ASF_MASK_DETECT;
int ftCode = ftEngine.init(context, DetectMode.ASF_DETECT_MODE_VIDEO, DetectFaceOrientPriority.ASF_OP_90_ONLY, FaceConfig.RECOGNIZE_MAX_DETECT_FACENUM, ftEngineMask);
ftInitCode.postValue(ftCode);
frEngine = new FaceEngine();
int frEngineMask = FaceEngine.ASF_FACE_RECOGNITION;
if(FaceConfig.ENABLE_FACE_QUALITY_DETECT) { frEngineMask |= FaceEngine.ASF_IMAGEQUALITY; } int frCode = frEngine.init(context, DetectMode.ASF_DETECT_MODE_IMAGE, DetectFaceOrientPriority.ASF_OP_90_ONLY, 10, frEngineMask); frInitCode.postValue(frCode); Int flCode = -1; int flCode = -1;if (FaceConfig.ENABLE_LIVENESS) {
flEngine = new FaceEngine();
int flEngineMask = (livenessType == LivenessType.RGB ? FaceEngine.ASF_LIVENESS : (FaceEngine.ASF_IR_LIVENESS | FaceEngine.ASF_FACE_DETECT));
if (needUpdateFaceData) {
flEngineMask |= FaceEngine.ASF_UPDATE_FACEDATA;
}
flCode = flEngine.init(context, DetectMode.ASF_DETECT_MODE_IMAGE,
DetectFaceOrientPriority.ASF_OP_90_ONLY, FaceConfig.RECOGNIZE_MAX_DETECT_FACENUM, flEngineMask);
flInitCode.postValue(flCode);
LivenessParam livenessParam = new LivenessParam(FaceConfig.RECOMMEND_RGB_LIVENESS_THRESHOLD, FaceConfig.RECOMMEND_IR_LIVENESS_THRESHOLD);
flEngine.setLivenessParam(livenessParam);
}
if (ftCode == ErrorInfo.MOK && frCode == ErrorInfo.MOK && flCode == ErrorInfo.MOK) {
Constants.isInitEnt = true; }}Copy the codeFace registered
public FaceEntity registerJpeg(Context context, FaceImageResult.DataBean data) throws RegisterFailedException {
if(faceRegisterInfoList ! = null && faceRegisterInfoList.size() >= MAX_REGISTER_FACE_COUNT) { Log.e(TAG,"registerJpeg: registered face count limited "+ faceRegisterInfoList.size()); Throw new RegisterFailedException(throw new RegisterFailedException("registered face count limited");
}
Bitmap bitmap = ImageUtil.jpegToScaledBitmap( Base64.decode(data.getImage(), Base64.DEFAULT), ImageUtil.DEFAULT_MAX_WIDTH, ImageUtil.DEFAULT_MAX_HEIGHT);
bitmap = ArcSoftImageUtil.getAlignedBitmap(bitmap, true);
byte[] imageData = ArcSoftImageUtil.createImageData(bitmap.getWidth(), bitmap.getHeight(), ArcSoftImageFormat.BGR24);
int code = ArcSoftImageUtil.bitmapToImageData(bitmap, imageData, ArcSoftImageFormat.BGR24);
if(code ! = ArcSoftImageUtilError.CODE_SUCCESS) { throw new RuntimeException("bitmapToImageData failed, code is " + code);
}
returnregisterBgr24(context, imageData, bitmap.getWidth(), bitmap.getHeight(), data); } /** * used to register a photo face ** @param context Context object * @param bgr24 bgr24 data * @param width bgr24 width * @param height bgr24 height * @param Name Specifies the saved name. If it is empty, the timestamp * @ is usedreturn*/ public FaceEntity registerBgr24(Context Context, byte[] bgr24, int width, int height, String name,String idCard) {if(faceEngine == null || context == null || bgr24 == null || width % 4 ! = 0 || bgr24.length ! = width * height * 3) { Log.e(TAG,"registerBgr24: invalid params");
returnnull; } List<FaceInfo> faceInfoList = new ArrayList<>(); int code; synchronized (faceEngine) { code = faceEngine.detectFaces(bgr24, width, height, FaceEngine.CP_PAF_BGR24, faceInfoList); }if(code == ErrorInfo.MOK && ! faceInfoList.isEmpty()) { code = faceEngine.process(bgr24, width, height, FaceEngine.CP_PAF_BGR24, faceInfoList, FaceEngine.ASF_MASK_DETECT);if (code == ErrorInfo.MOK) {
List<MaskInfo> maskInfoList = new ArrayList<>();
faceEngine.getMask(maskInfoList);
if(! maskInfoList.isEmpty()) { int isMask = maskInfoList.get(0).getMask();if(isMask == maskinfo. WORN) {/* * Registration requirements without a mask */ log.e (TAG,"registerBgr24: maskInfo is worn");
returnnull; } } } FaceFeature faceFeature = new FaceFeature(); /* * The extractType parameter is extracttype.register, The value of the parameter mask for MaskInfo. NOT_WORN * / synchronized (faceEngine) {code = faceEngine. ExtractFaceFeature (bgr24, width, height, FaceEngine.CP_PAF_BGR24, faceInfoList.get(0), ExtractType.REGISTER, MaskInfo.NOT_WORN, faceFeature); } String userName = name == null ? String.valueOf(System.currentTimeMillis()) : name; // Save the registration result (registration map, characteristic data)ifRect cropRect = getBestRect(width, height, faceInfolist.get (0).getRect());if (cropRect == null) {
Log.e(TAG, "registerBgr24: cropRect is null");
returnnull; } cropRect.left &= ~3; cropRect.top &= ~3; cropRect.right &= ~3; cropRect.bottom &= ~3; String imgPath = getImagePath(userName); // Create a Bitmap of your avatar, Bitmap headBmp = getHeadImage(bgr24, width, height, faceInfolist.get (0).getOrient(), cropRect, ArcSoftImageFormat.BGR24); try { FileOutputStream fos = new FileOutputStream(imgPath); headBmp.compress(Bitmap.CompressFormat.JPEG, 100, fos); fos.close(); } catch (IOException e) { e.printStackTrace();returnnull; } // Data synchronization in memoryif(faceRegisterInfoList == null) { faceRegisterInfoList = new ArrayList<>(); } FaceEntity faceEntity = new FaceEntity(name,idCard, imgPath, faceFeature.getFeatureData(),0L); // Determine whether this person exists, if there is overwrite, otherwise add (solve the problem due to reset face deletion and registration colleagues)if (faceRegisterInfoList.contains(faceEntity)) {
faceRegisterInfoList.remove(faceEntity);
List<FaceEntity> faceEntities = GreendaoUtils.Companion.getGreendaoUtils().searchFaceForIdcard(idCard);
if (faceEntities == null || faceEntities.isEmpty()) {
long faceId = GreendaoUtils.Companion.getGreendaoUtils().insert(faceEntity);
faceEntity.setFaceId(faceId);
}else{ faceEntities.get(0).setFeatureData(faceFeature.getFeatureData()); GreendaoUtils.Companion.getGreendaoUtils().update(faceEntities.get(0)); }}else {
long faceId = GreendaoUtils.Companion.getGreendaoUtils().insert(faceEntity);
faceEntity.setFaceId(faceId);
}
faceRegisterInfoList.add(faceEntity);
return faceEntity;
} else {
Log.e(TAG, "registerBgr24: extract face feature failed, code is " + code);
returnnull; }}else {
Log.e(TAG, "registerBgr24: no face detected, code is " + code);
returnnull; }}Copy the codeThe face of a search
/** * Search ** @param faceFeature for incoming feature data * @return*/ public CompareResult getTopOfFaceLib(FaceFeature FaceFeature) {if (faceEngine == null || faceFeature == null || faceRegisterInfoList == null || faceRegisterInfoList.isEmpty()) {
return null;
}
long start = System.currentTimeMillis();
FaceFeature tempFaceFeature = new FaceFeature();
FaceSimilar faceSimilar = new FaceSimilar();
float maxSimilar = 0;
int maxSimilarIndex = -1;
int code = ErrorInfo.MOK;
synchronized (searchLock) {
for (int i = 0; i < faceRegisterInfoList.size(); i++) {
tempFaceFeature.setFeatureData(faceRegisterInfoList.get(i).getFeatureData());
code = faceEngine.compareFaceFeature(faceFeature, tempFaceFeature, faceSimilar);
if(faceSimilar.getScore() > maxSimilar) { maxSimilar = faceSimilar.getScore(); maxSimilarIndex = i; }}}if (maxSimilarIndex != -1) {
return new CompareResult(faceRegisterInfoList.get(maxSimilarIndex), maxSimilar, code, System.currentTimeMillis() - start);
}
return null;
}
Copy the codeTemperature measurement part (core code)
Temperature head we use USB connection temperature head, using a simple USB analog keyboard, temperature head to measure the temperature analog keyboard input to the terminal text box, the code monitor keyboard input to read the temperature. Of course, you can also connect the temperature measuring head through the serial port, and take the initiative to send instructions to operate the temperature measuring head. Here, I use the way of simulating the keyboard.
public class ReadTemperatureHelper { private StringBuffer mStringBufferResult; Private Boolean mCaps; private boolean isCtrl; // Case private OnReadSuccessListener OnReadSuccessListener; public ReadTemperatureHelper(OnReadSuccessListener onReadSuccessListener) { this.onReadSuccessListener = onReadSuccessListener; mStringBufferResult = new StringBuffer(); } public void analysisKeyEvent(KeyEvent event) {int keyCode = event.getKeyCode(); CheckLetterStatus (event); checkInputEnt(event);if (event.getAction() == KeyEvent.ACTION_DOWN) {
char aChar = getInputCode(event);
if(aChar ! = 0) { mStringBufferResult.append(aChar); } Log.i("123123"."keyCode:" + keyCode);
if(keyCode == keyevent.keycode_enter) {// Enter returns log.i ("123123"."dispatchKeyEvent:" + mStringBufferResult.toString());
String s = mStringBufferResult.toString();
// int i = s.lastIndexOf(":");
// String substring = s.substring(i);
// String[] s1 = substring.split("");
Log.i("123123"."Temperature is:" + s);
onReadSuccessListener.onReadSuccess(s.trim());
mStringBufferResult.setLength(0);
}
}
}
/**
* ctrl
*/
private void checkInputEnt(KeyEvent event) {
if (event.getKeyCode() == KeyEvent.KEYCODE_CTRL_LEFT) {
if (event.getAction() == KeyEvent.ACTION_DOWN) {
isCtrl = true;
} else {
isCtrl = false; }}} /** *shift* * @param keyEvent */ private void checkLetterStatus(keyEvent keyEvent) {int keyCode = keyevent.getKeyCode ();if (keyCode == KeyEvent.KEYCODE_SHIFT_LEFT || keyCode == KeyEvent.KEYCODE_SHIFT_RIGHT) {
if(keyevent.getAction () == keyevent.action_down) {// holdshiftKey uppercase mCaps =true;
} else{// lowercase mCaps =false; Param keyEvent * @param keyEvent * @return
*/
private char getInputCode(KeyEvent keyEvent) {
char aChar;
int keyCode = keyEvent.getKeyCode();
Log.i("TAGKEYCODE", keyCode + "");
if(keyCode >= keyevent.keycode_a && keyCode <= keyevent.keycode_z)//29< keyCode <54 {// letter aChar = (char) ((mCaps?'A' : 'a') + keyCode - KeyEvent.KEYCODE_A); / /}else if(keyCode >= keyevent.keycode_0 && keyCode <= keyevent.keycode_9) {// numberif(mCaps)// Whether to hold downshiftKey {// hold down the need to convert the number to the corresponding character switch (keyCode) {case KeyEvent.KEYCODE_0:
aChar = ') ';
break;
case KeyEvent.KEYCODE_1:
aChar = '! ';
break;
case KeyEvent.KEYCODE_2:
aChar = The '@';
break;
case KeyEvent.KEYCODE_3:
aChar = The '#';
break;
case KeyEvent.KEYCODE_4:
aChar = '$';
break;
case KeyEvent.KEYCODE_5:
aChar = The '%';
break;
case KeyEvent.KEYCODE_6:
aChar = A '^';
break;
case KeyEvent.KEYCODE_7:
aChar = '&';
break;
case KeyEvent.KEYCODE_8:
aChar = The '*';
break;
case KeyEvent.KEYCODE_9:
aChar = '(';
break;
default:
aChar = ' ';
break; }}else {
aChar = (char) ('0'+ keyCode - KeyEvent.KEYCODE_0); }}else{// Other symbols switch (keyCode) {case KeyEvent.KEYCODE_PERIOD:
aChar = '. ';
break;
case KeyEvent.KEYCODE_MINUS:
aChar = mCaps ? '_' : The '-';
break;
case KeyEvent.KEYCODE_SLASH:
aChar = '/';
break;
case KeyEvent.KEYCODE_STAR:
aChar = The '*';
break;
case KeyEvent.KEYCODE_POUND:
aChar = The '#';
break;
case KeyEvent.KEYCODE_SEMICOLON:
aChar = mCaps ? ':' : '; ';
break;
case KeyEvent.KEYCODE_AT:
aChar = The '@';
break;
case KeyEvent.KEYCODE_BACKSLASH:
aChar = mCaps ? '|' : '\ \';
break;
default:
aChar = ' ';
break; }}returnaChar; } public interface OnReadSuccessListener { void onReadSuccess(String temperature); }}Copy the codeListen for keyboard input events in the activity’s dispatchKeyEvent method
override fun dispatchKeyEvent(event: KeyEvent?) : Boolean {if (isReadTemp) {
read.analysisKeyEvent(event)
if(event!! .keyCode == KeyEvent.KEYCODE_ENTER) {return true}}return super.dispatchKeyEvent(event)
}
Copy the codeOpen door (relay core code)
public void openG() {
String status = "1";
try {
FileOutputStream fos = new FileOutputStream("/sys/exgpio/relay1");
fos.write(status.getBytes());
fos.close();
} catch (Exception e) {
e.printStackTrace();
}
String status1 = "0";
SystemClock.sleep(200);
try {
FileOutputStream fos = new FileOutputStream("/sys/exgpio/relay1"); fos.write(status1.getBytes()); fos.close(); } catch (Exception e) { e.printStackTrace(); }}Copy the codeQ&A
Project development and use of also met a lot of problems, I think it’s worth noting that there are two 1, complex outdoor environment, existing face recognition for a long time without problems The problem is not occasional problem, after repeated tests, a dim light outdoor, because open the ir infrared live test, heat light deficiencies cause living test does not pass
2. The outdoor environment leads to inaccurate temperature measurement. This problem is caused by the principle of infrared temperature measurement technology, because the outdoor temperature is too high or too low to ensure the accuracy of temperature measurement, or the temperature can not be measured. At present, there is no solution, the later will measure the whole face frame with the temperature of each point in the area, for certain compensation to take the average.
supplement
The above simple list of some core code blocks, followed by the source code, the source code has a detailed business code, including reading ID card and scanning code, because the ID card reader is the company’s product, and other ID card reader card SDK is not the same, so the card reader SDK was deleted, the business code is retained.
After reading the ID card, go back to the background to verify the health code status of the person, and then determine whether to open the door
Read the health code through the serial port, and the code is written. After reading the health code, verify the status of the health code in the background to confirm whether the door is open
Because the test needs, so part of the code of health code is commented out, and the temperature given randomly in the project is easy to test
There is a lot of logic and things not written in the source code, get the source code directly
Welcome to exchange access source code access source code
To learn more about face recognition products, please visitRainbow soft visual open platformoh