Want to know what your future baby will look like?

Abstract: This case can be based on a father and mother’s face photo, to generate a photo of the child, and can adjust the parameters to see the appearance of children of different genders and ages.

This article is from The huawei cloud community “BabyGAN: Generating a Child’s photo from a parent’s photo”, by the light of Shan Hai.

This case can be based on a father and mother’s face photo, generate a photo of the child, and can adjust the parameters to see the appearance of children of different genders and ages.

When uploading photos of parents, try to upload photos that show their facial features and have a light background to ensure the effect of the photos.

This case is only for learning and communication. Do not use it for other purposes.

In addition, due to the imperfect technology, the generated child photos may be distorted or distorted, you can replace different parents photos, regenerate the child photos, until the satisfactory generation effect is achieved.

Let’s start by running the example step by step.

1. Install required modules

This step takes about 4 minutes

! pip install imutils moviepy dlibCopy the code

2. Download the code and model files

import os
import moxing as mox

root_dir = '/home/ma-user/work/ma_share/'
code_dir = os.path.join(root_dir, 'BabyGAN')
if not os.path.exists(os.path.join(root_dir, 'BabyGAN.zip')):
    mox.file.copy('obs://arthur-1/BabyGAN/BabyGAN.zip', os.path.join(root_dir, 'BabyGAN.zip'))
    os.system('cd %s; unzip BabyGAN.zip' % root_dir)

os.chdir(code_dir)
Copy the code

3. Load relevant modules and models

import cv2 import math import pickle import imageio import warnings import PIL.Image import numpy as np from glob import  glob from PIL import Image import tensorflow as tf from random import randrange import moviepy.editor as mpy import matplotlib.pyplot as plt from IPython.display import clear_output from moviepy.video.io.ffmpeg_writer import FFMPEG_VideoWriter import config import dnnlib import dnnlib.tflib as tflib from encoder.generator_model import Generator %matplotlib inline warnings.filterwarnings("ignore")Copy the code

Load the model file. This code block can only be executed once. If an error occurs, restart the kernel and run all the code again

tflib.init_tf() URL_FFHQ = "./karras2019stylegan-ffhq-1024x1024.pkl" with dnnlib.util.open_url(URL_FFHQ, cache_dir=config.cache_dir) as f: generator_network, discriminator_network, Gs_network = pickle.load(f) generator = Generator(Gs_network, batch_size=1, randomize_noise=False) model_scale = int(2 * (math.log(1024, 2) - 1)) age_direction = np.load('./ffhq_dataset/latent_directions/age.npy') horizontal_direction = np.load('./ffhq_dataset/latent_directions/angle_horizontal.npy') vertical_direction = np.load('./ffhq_dataset/latent_directions/angle_vertical.npy') eyes_open_direction = np.load('./ffhq_dataset/latent_directions/eyes_open.npy') gender_direction = np.load('./ffhq_dataset/latent_directions/gender.npy') smile_direction = np.load('./ffhq_dataset/latent_directions/smile.npy') def get_watermarked(pil_image: Image) -> Image: try: image = cv2.cvtColor(np.array(pil_image), cv2.COLOR_RGB2BGR) (h, w) = image.shape[:2] image = np.dstack([image, Np.ones ((h, w), dtype="uint8") * 255) cv2.IMREAD_UNCHANGED) (fwH, WH = int(PCT * h * 2) wW = int((wH * fwW)/fwH * 0.1) watermark = cv2.resize(full_watermark, (wH, wW), interpolation=cv2.INTER_AREA) overlay = np.zeros((h, w, 4), dtype="uint8") (wH, wW) = watermark.shape[:2] overlay[h - wH - 10: h - 10, 10: Overlay = image.copy(overlay, 0.5, output, 1.0, 0) output) rgb_image = cv2.cvtColor(output, cv2.COLOR_BGR2RGB) return Image.fromarray(rgb_image) except: return pil_image def generate_final_images(latent_vector, direction, coeffs, i): new_latent_vector = latent_vector.copy() new_latent_vector[:8] = (latent_vector + coeffs * direction)[:8] new_latent_vector = new_latent_vector.reshape((1, 18, 512)) generator.set_dlatents(new_latent_vector) img_array = generator.generate_images()[0] img = PIL.Image.fromarray(img_array, 'RGB') if size[0] >= 512: img = get_watermarked(img) img_path = "./for_animation/" + str(i) + ".png" img.thumbnail(animation_size, PIL.Image.ANTIALIAS) img.save(img_path) face_img.append(imageio.imread(img_path)) clear_output() return img def generate_final_image(latent_vector, direction, coeffs): new_latent_vector = latent_vector.copy() new_latent_vector[:8] = (latent_vector + coeffs * direction)[:8] new_latent_vector = new_latent_vector.reshape((1, 18, 512)) generator.set_dlatents(new_latent_vector) img_array = generator.generate_images()[0] img = PIL.Image.fromarray(img_array, 'RGB') if size[0] >= 512: img = get_watermarked(img) img.thumbnail(size, PIL.Image.ANTIALIAS) img.save("face.png") if download_image == True: files.download("face.png") return img def plot_three_images(imgB, fs=10): f, axarr = plt.subplots(1, 3, figsize=(fs, fs)) axarr[0].imshow(Image.open('./aligned_images/father_01.png')) axarr[0].title.set_text("Father's photo") axarr[1].imshow(imgB) axarr[1].title.set_text("Child's photo") axarr[2].imshow(Image.open('./aligned_images/mother_01.png')) axarr[2].title.set_text("Mother's photo") plt.setp(plt.gcf().get_axes(), xticks=[], yticks=[]) plt.show()Copy the code

4. Prepare pictures of mom and Dad

In this case, a default parent photo has been prepared. In the file Resource Management window in the left sidebar, go to the ma_share/BabyGAN directory and then go to the father_image or mother_image directory to see the provided parent photo, as shown below:

If you need to change photos of your parents, see section 11 of this article, “Changing Photos of Your parents.”

if len(glob(os.path.join('./father_image', '*.jpg'))) ! = 1 or (not os.path.exists('./father_image/father.jpg')): Raise Exception(' Please prepare a photo of your father under ma_share/BabyGAN/father_image, And named father. JPG ') if len (glob (OS) path) join ('/mother_image ', '*.jpg)))! = 1 or (not os.path.exists('./mother_image/mother.jpg')): Raise Exception(' Please prepare a photo of your mother in ma_share/BabyGAN/father_image and name it mother.jpg')Copy the code

5. Capture the father’s face area and align it

! python align_images.py ./father_image ./aligned_imagesCopy the code

Check the father’s face

if os.path.isfile('./aligned_images/father_01.png'):
    pil_father = Image.open('./aligned_images/father_01.png')
    (fat_width, fat_height) = pil_father.size
    resize_fat = max(fat_width, fat_height) / 256
    display(pil_father.resize((int(fat_width / resize_fat), int(fat_height / resize_fat))))
else:
    raise ValueError('No face was found or there is more than one in the photo.')
Copy the code

6. Capture the mother’s face area and align it

! python align_images.py ./mother_image ./aligned_imagesCopy the code

Look at the mother’s face

if os.path.isfile('./aligned_images/mother_01.png'):
    pil_mother = Image.open('./aligned_images/mother_01.png')
    (mot_width, mot_height) = pil_mother.size
    resize_mot = max(mot_width, mot_height) / 256
    display(pil_mother.resize((int(mot_width / resize_mot), int(mot_height / resize_mot))))
else:
    raise ValueError('No face was found or there is more than one in the photo.')
Copy the code

7. Extracting facial features

This step takes about 3 minutes

! Py \ --early_stopping False \ --lr=0.25 \ --batch_size=2 \ --iterations=100 \ --output_video=False \ ./aligned_images \ ./generated_images \ ./latent_representations if len(glob(os.path.join('./generated_images', '*.png'))) == 2: first_face = np.load('./latent_representations/father_01.npy') second_face = np.load('./latent_representations/mother_01.npy') print("Generation of latent representation is complete! Now comes the fun part.") else: raise ValueError('Something wrong. It may be impossible to read the face in the photos. Upload other photos and try again.')Copy the code

8. Generate a photo of a family of three

Modify the gender_influence and person_age parameters in the code below

Gender_influence: Gender influencing factor (range: 0.01, 0.99). The closer the value is to 0, the greater the influence of the father’s appearance and the greater the influence of the mother’s appearance.

Person_age: age influence factor, value range [10, 50], set this value, will generate the face of the corresponding age of the child.

Each time you change the value of this parameter, rerun the following code block to generate a new photo of your child

Genes_influence = 0.8 # gender influence factor, value range [0.01, 0.99], the closer the value is to 0, the greater the influence of the father's appearance, conversely, the greater the influence of the mother. Person_age = 10 # age influence factor, value range [10, 50], set this value, Style = "Default" if style == "Father's photo": lr = ((np.arange(1, model_scale + 1) / model_scale) ** genes_influence).reshape((model_scale, 1)) rl = 1 - lr hybrid_face = (lr * first_face) + (rl * second_face) elif style == "Mother's photo": lr = ((np.arange(1, model_scale + 1) / model_scale) ** (1 - genes_influence)).reshape((model_scale, 1)) rl = 1 - lr hybrid_face = (rl * first_face) + (lr * second_face) else: hybrid_face = ((1 - genes_influence) * first_face) + (genes_influence * second_face) intensity = -((person_age / 5) - 6)  resolution = "512" size = int(resolution), int(resolution) download_image = False face = generate_final_image(hybrid_face, age_direction, intensity) plot_three_images(face, fs=15)Copy the code

9. Check your child’s age range

Please modify the gender_influence parameter in the code below. The gender_influence parameter ranges from 0.01 to 0.99. The closer the value is to 0, the greater the influence of the father’s appearance is, while the influence of the mother is.

Each time you change the value of this parameter, rerun the following code block

Gender_influence = 0.8 # (range: 0.01, 0.99). The closer the value is to 0, the greater the influence of the father's appearance is, and the greater the influence of the mother's appearance is! rm -rf ./for_animation ! mkdir ./for_animation face_img = [] hybrid_face = ((1 - gender_influence) * first_face) + (gender_influence * second_face) animation_resolution = "512" animation_size = int(animation_resolution), int(animation_resolution) frames_number = 50 download_image = False for i in range(0, frames_number, 1): intensity = (8 * (i / (frames_number - 1))) - 4 generate_final_images(hybrid_face, age_direction, intensity, i) clear_output() print(str(i) + " of {} photo generated".format(str(frames_number))) for j in reversed(face_img): Face_img. appEnd (J) Automatic_download = False if gender_influence <= 0.3: Animation_name = "boy.mp4" Elif gender_influence >= 0.7: animation_name = "girl.mp4" else: animation_name = "boy.mp4" Elif gender_influence >= 0.7: animation_name = "girl.mp4" else: animation_name = "animation.mp4" imageio.mimsave('./for_animation/' + animation_name, face_img) clear_output() display(mpy.ipython_display('./for_animation/' + animation_name, height=400, autoplay=1, loop=1))Copy the code

10. Check your child’s gender

Modify the person_age parameter in the code below, which is an age-influencing factor in the range [10, 50], to generate the face of a child of that age.

Each time you change the value of this parameter, rerun the following code block

Person_age = 10 # The age of the child. The value ranges from 10 to 50. When set to this value, the face of the child will be generated. rm -rf ./for_animation ! mkdir ./for_animation face_img = [] intensity = -((person_age / 5) - 6) animation_resolution = "512" animation_size = Int (animation_resolution), int(animation_resolution) frames_number = 50 # Value range [10, 50] Download_image = False for I in range(1, frames_number): gender_influence = i / frames_number hybrid_face = ((1 - gender_influence) * first_face) + (gender_influence * second_face) face = generate_final_images(hybrid_face, age_direction, intensity, i) clear_output() print(str(i) + " of {} photo generated".format(str(frames_number))) for j in reversed(face_img): face_img.append(j) animation_name = str(person_age) + "_years.mp4" imageio.mimsave('./for_animation/' + animation_name, face_img) clear_output() display(mpy.ipython_display('./for_animation/' + animation_name, height=400, autoplay=1, loop=1))Copy the code

11. Change photos of mom and Dad

Next, you can upload the parent photos you’re interested in to the father_image and mother_image directories and run the code again to generate a new one.

You need to follow these rules and procedures:

1. Go to the ma_share/BabyGAN directory by referring to the following figure.

2. Upload a photo of your father to father_image as father.jpg. (If you do not know how to upload files to JupyterLab, please check this document)

3. Upload a photo of your mother to the mother_image directory and name it mother.jpg.

Father_image and mother_image directories both allow only one photo;

5. Re-run the code in Steps 4 to 10.

Click to follow, the first time to learn about Huawei cloud fresh technology ~