ZeroLess-DARTS: Improved differentiable architecture search with refined search operation and early stopping

Abstract

Differentiable architecture search (DARTS) method has gained noticeable popularity in neural architecture search (NAS) domain as it reduces the required search time compared to reinforcement learning and evolutionary based NAS algorithms. However, some further studies have indicated that the search algorithm of DARTS may be suboptimal, and its performance may deteriorate over the search process. In this paper, we provided a detailed performance analysis of the DARTS search algorithm (on the CIFAR10 image classification task) from different aspects such as changes in accuracies of derived architectures at each search epoch, the trend of changes in strengths of different operations over successive epochs, and the number of skip connections per normal cells. We propose ZeroLess-DARTS that considerably improves original DARTS performance on the CIFAR10 dataset (Cohen's D = 2.213), by refining the operation space in the search procedure and introducing an early stopping criterion. We show that our approach is generalizable to time series classification tasks by evaluating the performance of our model on one-dimensional ECG signals for WCT (Wide Complex Tachycardia) classification (Cohen's D = 1.706).

RAS ID

62737

Document Type

Conference Proceeding

Date of Publication

2-17-2023

School

School of Science

Copyright

subscription content

Publisher

Association for Computer Machinery

Comments

Fayyazifar, N., Ahderom, S., Samadiani, N., Maiorana, A., & Dwivedi, G. (2023). ZeroLess-DARTS: Improved differentiable architecture search with refined search operation and early stopping. In ICMLC '23: Proceedings of the 2023 15th International Conference on Machine Learning and Computing (pp. 54-60). Assocaitional for Computing Machinery. https://doi.org/10.1145/3587716.3587725

Link to Full Text

Share

 
COinS
 

Link to publisher version (DOI)

10.1145/3587716.3587725