Optimizing Generative Adversarial Network (GAN) Models for Non-Pneumatic Tire Design

Abstract

Pneumatic tires are used in diverse industries. However, their design is difficult, as it relies on the knowledge of experienced designers. In this paper, we generate images of non-pneumatic tire designs with patterns based on shapes and lines for different generative adversarial network (GAN) models and test the performance of the models. Using OpenCV, 2000 training images were generated, corresponding to spoke, curve, triangle, and honeycomb non-pneumatic tires. The images created for training were used after removing highly similar images by applying mean squared error (MSE) and structural similarity index (SSIM). To identify the best model for generating patterns of regularly shaped non-pneumatic tires, GAN, deep convolutional generative adversarial network (DCGAN), StarGAN v2, StyleGAN v2-ADA, and ProjectedGAN were compared and analyzed. In the qualitative evaluation, the GAN, DCGAN, StarGAN v2, and StyleGAN v2-ADA models distorted the circle shape and did not maintain a consistent pattern, but ProjectedGAN retained consistency in the circle, and the pattern was less distorted than in the other GAN models. When evaluating quantitative metrics, ProjectedGAN performed the best among several techniques when the difference between the generated and actual image distributions was measured.

1. Introduction

Pneumatic tires were invented in England in the late 1800s [1]. However, due to the limitations and problems of pneumatic tires, punctures can occur while driving and maintaining the proper amount of air pressure is essential and requires regular inspection and maintenance [2]. To solve these problems, non-pneumatic tires were introduced by creating polygonal elastomer spokes that replace and fill the air between the wheel and the tread [3].

Non-pneumatic tires are extremely durable because they cannot be punctured by air pressure. They are expected to be used in a variety of transportation vehicles, including military vehicles, forklifts, and lunar rovers, that travel over rough terrain [4]. However, non-pneumatic tires are widely used in construction and agriculture, but their design requires the expertise of skilled professionals with years of experience, and finding structured rules is difficult, making full or partial automation difficult [5]. Therefore, we propose to design non-pneumatic tires using GAN AI models.

Advances in AI technology have led to a wide range of applications in fields as diverse as image generation and conversion, speech synthesis, and natural language processing. These techniques are being utilized to solve many previously unsolvable problems by endowing computers with human-like cognitive abilities. Among them, GANs [6], generative models based on unsupervised learning, are making breakthroughs in generating images. For example, NVIDIA’s StyleGAN is a technique for creating realistic human faces based on GANs [7]. These techniques have applications in design [8], art [9], biotechnology [10], and the gaming industry [11]. Although GAN technology has exhibited high performance in image generation, it still has many limitations, including Nash equilibrium [12], gradient vanishing [13], mode collapse [14], mode dropping [15], a lack of diversity [16], and internal covariate shift [17]. To overcome these limitations, various GAN-based AI models, such as DCGAN [18], StarGAN v2 [19], StyleGAN v2-ADA [20], and ProjectedGAN [21], have been proposed.

To evaluate these GAN-based AI models, Inception Score (IS) [14], which measures diversity and quality; Fréchet Inception Distance (FID) [22], which measures the difference between generated and true image distributions; learned perceptual image patch similarity (LPIPS) [23], which measures the similarity of images by mimicking human visual perception; and precision and recall (PR) [24], which measures the precision and recall between generated and true images, have been utilized.

In this study, 2000 pattern images were used to train a GAN-based AI model. For the objective evaluation of GAN, DCGAN, StarGAN v2, StyleGAN v2-ADA, and ProjectedGAN, we used FID and LPIPS, which are commonly used evaluation indicators, and conducted quantitative performance evaluations of the models and qualitative design evaluations such as patterns and shape circles. From this, we conducted a comparative analysis of the objectivity inherent in evaluation indicators and the proficiency of the models in generating target images. This study is relevant to designers of non-pneumatic tires, who will be able to complete design work very efficiently and rapidly.

2. Test Methods

2.1. Acquiring Data

In this paper, we classified the shapes of non-pneumatic tires into four main categories using original images from the Korea Automotive Technology Institute (Katech) and various papers [25,26,27,28]. All images used for training were generated, not resized versions of original images or images found in papers. The images provided by Katech can be seen in Figure 1.

To compare the performance of the GAN models, we used OpenCV to generate 2000 images of the data to be trained. A total of 500 images were generated for each spoke shape, as shown in Figure 2.

Figure 2 shows sample data from a spoke design generated using OpenCV. Four types of non-pneumatic tire spokes were selected and generated: plate spokes, curved spokes, honeycomb spokes, and triangular spokes. The plate spoke in Figure-(a) is a straight, columnar structure that uses vertical stiffness to support weight. The curved spoke in Figure-(b) is a curved cell structure with an ammonite pattern that has a high compressive strength-to-volume ratio. The honeycomb spoke in Figure-(c) has a hexagonal cell structure that is flexible in both the vertical and shear directions and is currently the most commonly researched. The triangular spoke in Figure-(d) has a triangular cell structure and similar characteristics to the honeycomb spoke. We generated 500 images of each spoke design, corresponding to a total of 2000 images for training.

The initial image of the tire that was generated was 1024 × 1024, but it required a significant amount of time and resources for training. Additionally, resizing it to 256 × 256 did not yield significantly different results. Therefore, we unified the extension of each data point to PNG for easy training and resized the image to 256 × 256 pixels. We then compared the similarities of the generated images by calculating the differences in luminance, contrast, structure, and pixel values using SSIM [29] and MSE [30] algorithms: SSIM compares the luminance, contrast, and structure of the original image and the comparison image, whereas MSE shows the measured value of the difference in each pixel value. Similar images were removed based on the comparison of the similarity of the images, and the remaining images were used as data for training.

2.2. GAN

GANs [6] have received continuous attention from deep learning researchers due to their emphasis on high-quality image generation, diversity of generated images, and stability of training. This is because GANs are models that can be applied to a variety of fields, including computer vision, natural language processing, time series synthesis, and semantic segmentation, but they have been particularly successful in generating synthetic image data.

As shown in Figure 3, a GAN is a game-theoretic AI model that uses two convolutional neural networks (CNNs) [31], each acting as a generator to produce an image and a discriminator to determine whether the image is real or fake and attempts to find a Nash equilibrium between the two CNNs. In a GAN, the generator and discriminator compete by generating data that resemble real data, and a sample that cannot be determined to be real or fake, considered a perfect equilibrium, is the optimal outcome. A GAN is divided into two parts: the part that identifies real images and the part that identifies false (fake) images generated by the constructor, as shown in Equation (1).

$$\underset{\mathrm{G}}{\mathrm{Min}}\underset{\mathrm{D}}{\max }\mathrm{V}\left(\mathrm{D},\mathrm{G}\right)={\mathbb{E}}_{\mathrm{x}~{\mathrm{p}}_{\mathrm{d}\mathrm{a}\mathrm{t}\mathrm{a}}\left(\mathrm{x}\right)}\left[\mathrm{l}\mathrm{o}\mathrm{g}\mathrm{D}\left(\mathrm{x}\right)\right]+{\mathbb{E}}_{\mathrm{z}~{\mathrm{p}}_{\mathrm{z}}\left(\mathrm{z}\right)}\left[\log \left(1-\mathrm{D}\left(\mathrm{G}\left(\mathrm{z}\right)\right)\right)\right]$$

(1)

${\mathbb{E}}_{\mathrm{x}~{\mathrm{p}}_{\mathrm{d}\mathrm{a}\mathrm{t}\mathrm{a}}\left(\mathrm{x}\right)}\left[\mathrm{l}\mathrm{o}\mathrm{g}\mathrm{D}\left(\mathrm{x}\right)\right]$ is the part where the actual image x is put into D(x) to increase the log value so that it has a high probability value.${\mathbb{E}}_{\mathrm{z}~{\mathrm{p}}_{\mathrm{z}}\left(\mathrm{z}\right)}\left[\log \left(1-\mathrm{D}\left(\mathrm{G}\left(\mathrm{z}\right)\right)\right)\right]$ is the part where the image generated by G(x) is put into D(x) to make the log value smaller, resulting in a lower probability value. The constructor plays a min/max game, learning to produce images that are as close to the real image as possible until the discriminator fails to distinguish between the real and fake images.

For the original GAN, we used only linear layers for training because they were easier to model and faster to learn. We fixed the learning rate to 0.0002 for the discriminator and 0.0001 for the generator and applied the Adam optimizer [32] to a batch size of 128 and 256 × 256-pixel-size images for 2000 epochs. The result is shown in Figure 4. The tires and wheelbase appeared to be consistent with no distortion, but the image quality was very low, making it difficult to see the shape of the spoke design, and the FID values were not calculated due to mode collapse, which resulted in the continual production of similar images, reducing diversity.

2.3. DCGAN

DCGAN [18] is a model that combines the general GAN model with four learning techniques employed in CNN models: all convolutional net (All-CNN) [33], eliminating fully connected layers [34], batch normalization [35], and ReLU [36] + LeakyReLU [37].

• All convolutional net: Introduced in 2014, All-CNN is a method that removes all deterministic spatial pooling functions [38], such as maxpolling [39], that are applied between the layers of a CNN. The layers are constructed using only strided convolutions without any pooling functions, allowing for self-downsampling.
• Eliminating fully connected layers: This method does not utilize fully connected layers and consists of all convolutional layers. DCGAN directly connects the top convolutional features of the constructor and the discriminator to make the model stable and increase the speed at which it converges.
• Batch normalization: Batch normalization produces a mean of 0 and a variance of 1 in each batch of data. This makes the model less sensitive to scale and, therefore, more stable during learning. This method helps to overcome the problem of poor learning due to poor initial weights or poor updating of the gradient descent method [35,40]. BatchNorm has been shown to play an important role in preventing the mode collapse of all samples to a single point, which is a common problem with GANs, and in promoting deeper generator layers. However, applying BatchNorm to all layers may result in the model repeating only a few patterns with limited samples or instability. Therefore, we did not apply BatchNorm to the output layer of the generator or the input layer of the discriminator.
• ReLU + LeakyReLU: We used Tanh activation [41] for the output layer of the generator and ReLU activation for all other layers. For the discriminator, we used LeakyReLU activation to maintain a high resolution.

Based on these four fundamental methods, we improved the original GAN, changed the structure of the model to produce higher quality images, and used batch normalization and ReLU activation to improve the training stability and performance.

We used 2000 images of four types, created using OpenCV, for training. Because the existing DCGAN has an architecture optimized for 64 × 64 images, to perform training for the 256 × 256 image size used in this study, we added another convolutional layer to the existing architecture, as shown in Figure 5. The architecture is based on the DCGAN code provided by Pytorch [42].

Figure 6 shows the training results of the DCGAN, which generated image sizes of 256 × 256. The DCGAN shows an increase in the quality and diversity of images compared to the original GAN. However, noise in the background and an inability to generate consistent patterns is also evident. Finally, we generated 2000 dummy images and compared them with the real images, resulting in an FID of 192.02 and an LPIPS of 0.4986.

2.4. StarGAN

An image-to-image translation model must preserve two properties: image diversity and scalability across multiple domains. However, existing models either synthesize images using only two domains or require a corresponding number of generators if there are many domains. For example, if there are K domains, existing methods require K(K − 1) generators and a training effort that depends on their number. StarGAN v2 proposes four new frameworks to meet these two requirements.

Generator: The generator (G) takes an image x as input and outputs G(x,s), reflecting a style code (s). The style code (s) is provided by a mapping network (F) or a style encoder (E). s is used in G by applying AdaIN [43] to it. This method eliminates the need to input numerous y directly into G and allows G to synthesize images from any domain. A description of these notation can be found in

Table 1. Notation used in StarGAN v2.

Notation	Denote
$\mathrm{G}$	Generator
$\mathrm{D}$	Discriminator
$\mathrm{s}$	Style coder
$\mathrm{E}$	Style Encoder
$\mathrm{F}$	Mapping Network
$\mathrm{z}$	Latent code
$\mathrm{y}$	domain
$\mathrm{x}$	Input Image

.

Mapping network: The mapping network (F) takes a latent code (z) and a domain (y) as input and generates a style code $\mathrm{s}={\mathrm{F}}_{\mathrm{y}}\left(\mathrm{z}\right)$. ${\mathrm{F}}_{\mathrm{y}}$ is characterized for domain y. F consists of an MLP with multiple output branches and provides style codes for all domains. F takes a random latent vector z and samples from domain y to generate a set of style codes.

Style encoder: E takes an image x that matches y as input. E extracts the style code of x, $\mathrm{s}={\mathrm{E}}_{\mathrm{y}}\left(\mathrm{x}\right)$. It is similar to the mapping network F, but because E takes a random number of images rather than a latent vector as input, a style code that references an image can be obtained.

Discriminator: The discriminator (D) in StarGAN v2 is a multi-task discriminator (D) consisting of several output branches. Each branch ${\mathrm{D}}_{\mathrm{y}}$ learns a binary classification that distinguishes between a real image x in domain y and a fake image G(x,s) provided by the generator (G).

Images x and $\tilde{\mathrm{s}}$ are input into G using Equation (2) and trained to output $\mathrm{G}\left(\mathrm{x},\tilde{\mathrm{s}}\right)$ with adversarial loss.

$${\mathrm{L}}_{\mathrm{a}\mathrm{d}\mathrm{v}}={\mathbb{E}}_{\mathrm{x},\mathrm{y}}\left[\mathrm{l}\mathrm{o}\mathrm{g}{\mathrm{D}}_{\mathrm{y}}\left(\mathrm{x}\right)\right]+{\mathbb{E}}_{\mathrm{x},\tilde{\mathrm{y}},\mathrm{z}}\left[\log \left(1-{\mathrm{D}}_{\tilde{\mathrm{y}}}\left(\mathrm{G}\left(\mathrm{x},\tilde{\mathrm{s}}\right)\right)\right)\right]$$

(2)

where ${\mathrm{D}}_{\mathrm{y}}\left(·\right)$ denotes the output for domain y in D. In other words, if image x is in domain y, the output is fake for all other domains except the corresponding domain and real for that domain. F is trained to provide a style code $\tilde{\mathrm{s}}$ that is likely to be domain $\tilde{\mathrm{y}}$, and G is trained to use $\tilde{\mathrm{s}}$ to produce a $\mathrm{G}\left(\mathrm{x},\tilde{\mathrm{s}}\right)$ that is indistinguishable from the real image of domain $\tilde{\mathrm{y}}$.

Equation (3) is the style reconstruction loss. The loss reconstructs the original image x using the generated image $\mathrm{G}\left(\mathrm{x},\tilde{\mathrm{s}}\right)$ and the style code. It attempts to minimize the difference between the reconstructed image and the original image x. This allows G to learn from the input images x and s to produce an image of the desired domain.

$${\mathrm{L}}_{\mathrm{s}\mathrm{t}\mathrm{y}}={\mathbb{E}}_{\mathrm{x},\tilde{\mathrm{y}},\mathrm{z}}\left[{‖\tilde{\mathrm{s}}-{\mathrm{E}}_{\tilde{\mathrm{y}}}\left(\mathrm{G}\left(\mathrm{x},\tilde{\mathrm{s}}\right)\right)‖}_1\right]$$

(3)

Equation (4) is the diversity-sensitive loss that allows the generator to learn a variety of images.

$${\mathrm{L}}_{\mathrm{d}\mathrm{s}}={\mathbb{E}}_{\mathrm{x},\tilde{\mathrm{y}},\mathrm{z}1,\mathrm{z}2}\left[{‖\mathrm{G}\left(\mathrm{x},{\tilde{\mathrm{s}}}_1\right)-\mathrm{G}\left(\mathrm{x},{\tilde{\mathrm{s}}}_2\right)‖}_1\right]$$

(4)

Here, the provided latent codes z1 and z2 are used by F to generate style codes ${\tilde{\mathrm{s}}}_1$ and ${\tilde{\mathrm{s}}}_2$. The corresponding loss maximizes the regularization term, which encourages G to generate different styles and explore a wider image space.

Equation (5) is the cycle consistency loss, which ensures that the common characteristics of images belonging to a particular domain are preserved.

$${\mathrm{L}}_{\mathrm{c}\mathrm{y}\mathrm{c}}={\mathbb{E}}_{\mathrm{x},\mathrm{y},\tilde{\mathrm{y}},\mathrm{z}}\left[{‖\mathrm{x}-\mathrm{G}(\mathrm{G}\left(\mathrm{x},\tilde{\mathrm{s}}\right).\widehat{\mathrm{s}}‖}_1\right]$$

(5)

where $\widehat{\mathrm{s}}={\mathrm{E}}_{\mathrm{y}}\left(\mathrm{x}\right)$ is the style code of the input image x with y as the original domain of x. In other words, $\widehat{\mathrm{s}}$ generates a new style code that considers the domain of the input image rather than the style code extracted from the original domain. By using this $\widehat{\mathrm{s}}$ to reconstruct the input image x, G can perform a style-consistent transformation while preserving the original properties of x.

The four formulas can be summarized as follows:

$$\underset{\mathrm{G},\mathrm{F},\mathrm{E}}{\min }\underset{\mathrm{D}}{\max }={\mathrm{L}}_{\mathrm{a}\mathrm{d}\mathrm{v}}+{\mathsf{\lambda }}_{\mathrm{s}\mathrm{t}\mathrm{y}}{\mathrm{L}}_{\mathrm{s}\mathrm{t}\mathrm{y}}-{\mathsf{\lambda }}_{\mathrm{d}\mathrm{s}}{\mathrm{L}}_{\mathrm{d}\mathrm{s}}+{\mathsf{\lambda }}_{\mathrm{c}\mathrm{y}\mathrm{c}}{\mathrm{L}}_{\mathrm{c}\mathrm{y}\mathrm{c}}$$

(6)

In Equation (6), ${\mathsf{\lambda }}_{\mathrm{s}\mathrm{t}\mathrm{y}},{\mathsf{\lambda }}_{\mathrm{d}\mathrm{s}},{\mathsf{\lambda }}_{\mathrm{c}\mathrm{y}\mathrm{c}}$ are the hyperparameters for each loss. When training the model, instead of using only latent vectors, we further trained it by using images to generate style codes. This approach yielded better style transfer.

In conclusion, StarGAN v2 is an image transformation model that can handle different domains in a single dataset and constructor through the four processes described above. The domain here refers to a set of data with the same image feature values. Compared to the existing models, StarGAN v2 is more effective in terms of computational efficiency and image quality.

Figure 7 shows the training results after changing only the settings of num_domains and the directory under the default settings of the training networks in StarGAN v2-ADA on Github [44]. Figure 7 shows the output of the synthesis of the first and second rows in the third row. Noticeably, the shape of the lines are represented for the spoke- and triangle-type tires, whereas for the curve- and honeycomb-type tires, the features of the domain were not learned, and the image disappeared. Because the qualitative evaluation yielded meaningless results, we did not proceed with the quantitative evaluation, FID.

2.5. StyleGAN v2-ADA

StyleGAN v2-ADA is an adaptive discriminator augmentation mechanism for solving the problem of overfitting with less data [45]. To train for modern high-resolution high-quality images, 10⁵ to 10⁶ images are required. If overfitting occurs with fewer training images, the feedback from the discriminator becomes meaningless, and the learning results begin to diverge. To solve this problem, data augmentation is used during deep learning to prevent overfitting. For example, rotating an image or applying noise to it to train an image classifier increases the invariance of the image to semantic-preserving distortions and increases the performance of the image classifier. These image variations can be used to make the images generated by full GAN more realistic and diverse. However, augmentation leakage can occur where the distribution of the augmented images follows the distribution of the original dataset, producing images with different characteristics. To avoid the above problems, StyleGAN v2-ADA proposes four solutions.

Stochastic discriminator augmentation: StyleGAN v2-ADA is a model with an architecture similar to that of bCR [46]. However, in the StyleGAN v2-ADA architecture, we removed the CR loss term from bCR and modified the model so that the discriminator uses only the images with augmentation applied. The generator also learns using only the images with augmentation applied. We call this approach stochastic discriminator augmentation.

Design augmentations that do not leak: If the data are augmented or corrupted to allow an invertible transformation, the discriminator can find the distribution of normal images from the corrupted images [47]. This can be achieved by making the images with normal distributions non-leaking with an appropriate fraction (p < 1) of the augmented images.

Adaptive discriminator augmentation: ADA is a method of dynamically automating the control based on the degree of overfitting to reduce the effort and time for learning by manually adjusting the hyperparameters. The basic method of measuring overfitting is using a separate validation set and observing the difference from the training set. This method has the disadvantage of having to split the validation set even if small amounts of training data are available. However, using Equation (7), ADA can quantify overfitting without a validation set.

$${\mathrm{r}}_{\mathrm{v}}=\frac{\mathbb{E}\left[{\mathrm{D}}_{\mathrm{t}\mathrm{r}\mathrm{a}\mathrm{i}\mathrm{n}}\right]-\mathbb{E}\left[{\mathrm{D}}_{\mathrm{v}\mathrm{a}\mathrm{l}\mathrm{i}\mathrm{d}\mathrm{a}\mathrm{t}\mathrm{i}\mathrm{o}\mathrm{n}}\right]}{\mathbb{E}\left[{\mathrm{D}}_{\mathrm{t}\mathrm{r}\mathrm{a}\mathrm{i}\mathrm{n}}\right]-\mathbb{E}\left[{\mathrm{D}}_{\mathrm{g}\mathrm{e}\mathrm{n}\mathrm{e}\mathrm{r}\mathrm{a}\mathrm{t}\mathrm{e}\mathrm{d}}\right]} {\mathrm{r}}_{\mathrm{t}}=\mathbb{E}\left[\mathrm{s}\mathrm{i}\mathrm{g}\mathrm{n}\left({\mathrm{D}}_{\mathrm{t}\mathrm{r}\mathrm{a}\mathrm{i}\mathrm{n}}\right)\right]$$

(7)

Our enhancement pipeline: Given the diversity of images, we have organized our pipeline into 18 transformation types, which can be grouped into six categories: pixel blitting (x-flips, 90° rotations, and integer translations), general geometric transformations, color transformations, image-space filtering, additive noise, and cropping. We applied the transformations in a fixed order. The strength of the augmentation was controlled by a scalar p. We found that as long as p is kept below a safety threshold, the generator produces only clean images.

Figure 8 is the result of training for 2000 epochs using the Training New Networks command of StyleGAN v2-ADA with the hyperparameters set to default. When trained with StyleGAN v2-ADA, the image quality is significantly improved compared to that of the original GAN and DCGAN, and the pattern shape is relatively diverse. However, the circles were squashed, the tire shape was not properly visible, and the shape was not consistently generated. We used the FID calculation code built into StyleGAN v2-ADA and obtained an FID of 42.88 and an LPIPS of 0.3806.

2.6. Projected GAN

Projected GAN adds a feature projector, ${\mathrm{P}}_{\mathrm{l}}$, to the original GAN, as shown in Equation (8). ${\mathrm{P}}_{\mathrm{l}}$ maps the real image and the generated image into the input space of the discriminator.

$$\underset{\mathrm{G}}{\min }\underset{\left\{{\mathrm{D}}_{\mathrm{l}}\right\}}{\max }\sum _{\mathrm{l}\in \mathrm{L}}\left({\mathbb{E}}_{\mathrm{x}}\left[\mathrm{l}\mathrm{o}\mathrm{g}{\mathrm{D}}_{\mathrm{l}}\left({\mathrm{P}}_{\mathrm{l}}\left(\mathrm{x}\right)\right)\right]+{\mathbb{E}}_{\mathrm{z}}\left[\log \left(1-{\mathrm{D}}_{\mathrm{l}}\left({\mathrm{P}}_{\mathrm{l}}\left(\mathrm{G}\right(\mathrm{z}\left)\right)\right)\right)\right]\right)$$

(8)

${\mathrm{D}}_{\mathrm{l}}$ works on any independent feature projection. ${\mathrm{P}}_{\mathrm{l}}$ is fixed, and only the parameters of G and ${\mathrm{D}}_{\mathrm{l}}$ are optimized. By fixing ${\mathrm{P}}_{\mathrm{l}}$, no competition among the producers occurs. Thus, the variation in the producers is small, resulting in stable learning. The projected GAN proposed three main methods to speed up the convergence of learning to make it comparable to that of existing GANs and solve the problem of the discriminator not fully utilizing the pre-trained features of deeper layers.

Multi-scale discriminators: Multi-scale discriminators extract features from four layers, ${\mathrm{L}}_{\mathrm{l}}$, of a pre-trained feature network F. Each layer, ${\mathrm{L}}_{\mathrm{l}}$, is applied to a separate discriminator, ${\mathrm{D}}_{\mathrm{l}}$, using a simple convolutional architecture that uses spectral normalization [48]. We used a simple convolutional architecture, where each discriminator ${\mathrm{D}}_{\mathrm{l}}$ utilizes spectral normalization [48]. We found that the best performance was achieved by applying convolution until each discriminant had a scale of 4, 4 and then selecting logit. Finally, the logit values generated by each of the four discriminators were summed and used as the loss.

Random projections: We identified two strategies to make information about all features available to the discriminator. The first is cross-channel mixing (CCM) and the second is cross-scale mixing (CSM). CCM is a cross-channel mixing method that applies a 1 × 1 convolution with the same number of inputs, outputs, and channels so that the input is more informative. Increasing the number of output channels improves performance because the image information can be better preserved. CSM uses 3 × 3 convolution and bilinear interpolation to mix the CCM output with the CCM output. These two methods allow us to preserve large amounts of information with fewer layers and the performance is better than that of traditional GANs.

Pre-trained feature network: In this scheme, we experimented with different feature networks. The first, EfficientNets [49], allows for direct control of model size and performance; that is, by changing or removing the structure of the model, the size of the model can be reduced or increased, and the performance of the model changes accordingly. EfficientNets is an image classification model trained with ImageNet [50] that models the tradeoff between accuracy and computation. Second, we used ResNets [51] of different sizes. We also used ResNets trained on the R50-CLIP dataset, which is a dataset of image–text pairs, instead of ImageNet, to analyze how models pre-trained with features from ImageNet affect GAN. Finally, we used the vision transformer architecture (ViTBase) [52] and its successor (DeiT small distilled) [53]. We excluded the inception network [54] because it is used to extract feature maps from FID metrics, and the generated images can be correlated with FIDs.

Unlike traditional GAN models, Projected GAN is a method that trains a GAN model using a pre-trained situation. This increases data efficiency and removes unnecessary constraints to improve image quality, training speed, and sample utility. Although GAN produces high-quality images, it requires numerous datasets, high computing power, and careful handling of hyperparameters to perform smooth training. However, Projected GAN solves these problems by fixing the image data generated in the pre-training environment and the actual image data for training. Projected GAN exhibits high performance with very low FID values compared to other GAN models.

Figure 9 shows the results of training Projected GAN. As the number of epochs increased, the image quality and circle distortion improved, and the spoke design became denser and more detailed. Compared to other GANs, the projected GAN model performed the best among the GANs used so far, ideally generating multiple images with a variety of features, such as differences in the size of the circles on the spokes, the design of the spokes, and the thickness of the lines. The final FID was 13.41, which was the highest among the five models.

3. Validate Image Similarity

Metrics exist for evaluating the performance of the GAN model while verifying image similarity, such as IS, FID, and LPIPS. IS is a metric that evaluates the quality of the images generated by the GAN model, and this metric aims to evaluate how qualitatively good and diverse the generated images are. The better the IS, the higher the value is. In Equation (9), x in $\mathrm{x}~{\mathrm{p}}_{\mathrm{g}}$ denotes the sample of ${\mathrm{p}}_{\mathrm{g}}$. IS consists of ${\mathbb{E}}_{\mathrm{x}~{\mathrm{p}}_{\mathrm{g}}}$, which is the expected value; $\mathrm{p}\left(\mathrm{y}|\mathrm{X}\right)$, the conditional probability distribution; and p(y), the marginal class distribution. D_KL denotes the KL divergence (Kullback–Leibler divergence) [55] and measures the difference between two probability distributions, $\mathrm{p}\left(\mathrm{y}|\mathrm{x}\right)$ and p(y). $\mathrm{p}\left(\mathrm{y}|\mathrm{x}\right)$ is the conditional probability of a class and denotes the probability distribution of the class on the input data. In $\mathrm{p}\left(\mathrm{y}|\mathrm{x}\right)$, y refers to a class, and x refers to the input data. Therefore, it calculates the probability of an image being y given x. It calculates the probability of an image being in each class and places x in the class with the highest probability. $\mathrm{p}\left(\mathrm{y}\right)=\int \mathrm{p}\left(\mathrm{y}|\mathrm{x}\right){\mathrm{p}}_{\mathrm{g}}\left(\mathrm{x}\right)$ is the marginal class distribution. Marginal probability distributions are used to remove the effects of variables from a multivariate probability distribution and calculate the distribution of some variables. Marginal probability distributions are used to test for independence between variables and collapse variables, and determine the overall properties of a probability distribution.

$$\mathrm{I}\mathrm{S}\left(\mathrm{G}\right)=\exp \left({\mathbb{E}}_{\mathrm{x}~{\mathrm{p}}_{\mathrm{g}}}{\mathrm{D}}_{\mathrm{K}\mathrm{L}}\left(\mathrm{p}\left(\mathrm{y}|\mathrm{X}\right)\left|\right|\mathrm{p}\left(\mathrm{y}\right)\right)\right.$$

(9)

FID was developed to compensate for the shortcomings of IS and facilitate the evaluation of GAN performance. The drawback of IS is that it evaluates GANs by only considering the statistics of the generated images rather than the statistics of the real-world samples. In other words, IS only evaluates the quality and diversity of the generated images and does not directly compare the similarity between the generated and real-world images. Equation (10), on the other hand, measures the statistical similarity between the quality of the generated images and the real image by measuring the distance between two probability distributions. For FID, the lower the evaluation score, the better the performance of the model.

$$\mathrm{F}\mathrm{I}\mathrm{D}={‖{\mathsf{\mu }}_{\mathrm{r}}-{\mathsf{\mu }}_{\mathrm{g}}‖}^2+{\mathrm{T}}_{\mathrm{r}}\left(\sum _{\mathrm{r}}+\sum _{\mathrm{g}}-2\right({\sum _{\mathrm{r}}\sum _{\mathrm{g}})}^{\frac{1}{2}})$$

(10)

The learned perceptual image patch similarity (LPIPS) metric was created to address the inability of L2/PSNR [56], SSIM, and FSIM [57] to reflect human perceptual abilities. LPIPS utilizes the image classification models SqueezeNet [58], AlexNet [49], and VGG [59] to attempt to measure similarity based on human perception, in contrast to IS and FID. Given an image $\mathrm{x},{\mathrm{x}}_0$ in Equation (11), a pre-trained network of ImageNets [49] is fed an activation map ${\widehat{\mathrm{y}}}^{\mathrm{l}},\widehat{{\mathrm{y}}_0^{\mathrm{l}}}\in {\mathbb{R}}^{{\mathrm{H}}_{\mathrm{l}}\times {\mathrm{W}}_{\mathrm{l}}\times {\mathrm{C}}_{\mathrm{l}}}$ at layer l. The activation map is then fed to the network in Equation (11). The activation map is scaled by ${\mathsf{\omega }}_{\mathrm{l}}$, and the Euclidean distance [60] is then calculated. Finally, the LPIPS can be calculated by spatially averaging and summing per channel. The lower the LPIPS, the more similar they are.

$$\mathrm{d}\left(\mathrm{x},{\mathrm{x}}_0\right)=\sum _{\mathrm{l}}\frac{1}{{\mathrm{H}}_{\mathrm{l}}{\mathrm{W}}_{\mathrm{l}}}\sum _{\mathrm{h},\mathsf{\omega }}\left|\right|{\mathsf{\omega }}_{\mathrm{l}}\odot \left({\widehat{\mathrm{y}}}_{\mathrm{h}\mathrm{w}}^{\mathrm{l}}-{\widehat{\mathrm{y}}}_{0\mathrm{h}\mathrm{w}}^{\mathrm{l}}\right)\left|\right|$$

(11)

4. Results

The quantitative metric FID is an evaluation metric that verifies the statistical similarity of images by comparing the distance between the distribution of the actual image and the distribution of the generated image, and LPIPS is a method that compares the similarity of the characteristics of images through a model that reflects human perceptual abilities. Lists the qualitative and quantitative evaluation of five GAN models in

Table 2. Training results and evaluation metrics based on GAN models.

	Spoke	Curve	Triangle	Honeycomb	FID	LPIPS	Notes
Real					-	-	-
GAN					-	-	Mode collapse
DCGAN					192.02	0.4986	-
StarGAN v2					-	-	Mode collapse and failure to learn the domain
StyleGAN v2-ADA					42.88	0.3806	-
Projected GAN					13.41	0.3477	-

: GAN, DCGAN, StarGAN, StyleGAN v2, and ProjectedGAN, including the image synthesis model.

In the case of GAN, large amounts of noise were generated in the background image, and the shape of the pattern was barely visible. However, with less training data, mode collapse could be observed. We did not measure FID and LPIPS separately because the qualitative evaluation yielded meaningless results.

For DCGAN, we had the same background noise as GAN, but no mode collapse occurred, and we saw that it learned a variety of patterns. However, we had a problem mixing multiple patterns. The FID was 192.02, and the LPIPS was 0.4986.

StarGAN v2 failed to learn all domain types except for spoke, as listed in Table 2. For the curve type, the spoke appears to bend a little, but not in the form of a pattern. For the triangle and honeycomb types, no images were generated at all. We did not measure quantitative evaluation metrics because they were deemed meaningless through the qualitative evaluation.

StyleGAN v2-ADA generated high-quality, noise-free images with less data. However, it was unable to maintain the circle, resulting in distortion, and the patterns were distorted accordingly. However, compared to GAN and DCGAN, it produced more realistic images. The evaluation metrics FID and LPIPS were 42.88 and 0.3806, respectively, exhibiting a significant reduction compared to DCGAN.

The image generation from ProjectedGAN exhibited no noise in the background, and the resolution of the image was kept high. It also showed that the circle was not distorted, and the pattern was consistently drawn. In terms of qualitative evaluation, the FID was 13.41, exhibiting a reduction of approximately 30 compared to StyleGAN v2-ADA, and LPIPS was reduced to 0.3477, corresponding to the best result among the GANs used in this paper.

5. Conclusions

This study presents a survey of GAN models and their improvements and provides a detailed analysis of the application of patterns to GANs based on primitive shapes. At its core, this study addressed the fact that GAN models trained on complex image data such as people and Pokémon, which have been covered in previous studies, exhibit good performance in generating realistic images, but a large performance gap exists in generating images with simple shapes and regular patterns based on GAN models [61]. That is because achieving perfect symmetry requires simple shapes and regular patterns, which can be more challenging than training using humans and Pokémon. In practice, we found that GAN and DCGAN retained the circle but failed to learn the pattern domain and generated large amounts of noise in the background. StarGAN v2 learned the plate spoke domain but failed to learn the remaining domains. StyleGAN v2-ADA seemingly learned the patterns well but was unable to maintain circularity and suffered from distortion. In conclusion, we demonstrate that the image generated by ProjectedGAN best maintains the patterns based on primitive shapes, and the tire is neither circular nor distorted compared to other GANs. In keeping with the qualitative evaluation, the quantitative measures FID and LPIPS were 13.41 and 0.3477, respectively, which were superior among the four models.

Non-pneumatic tires are widely used in construction and agriculture, but their design requires expertise from skilled professionals with years of experience, and finding structured rules is difficult, making full or partial automation a challenge. In this study, we conducted a wide range of research on how to generate basic images of pneumatic tires in the most effective way using various techniques of artificial intelligence, which will greatly increase the efficiency of pneumatic tire design.