TY - GEN
T1 - The Two Dimensions of Worst-case Training and Their Integrated Effect for Out-of-domain Generalization
AU - Huang, Zeyi
AU - Wang, Haohan
AU - Huang, Dong
AU - Lee, Yong Jae
AU - Xing, Eric P.
N1 - Acknowledgements. This work was supported in part by NSF CAREER IIS-2150012 and IIS-2204808. HW was supported by NIH R01GM114311, NIH P30DA035778, and NSF IIS1617583.
PY - 2022
Y1 - 2022
N2 - Training with an emphasis on 'hard-to-learn' components of the data has been proven as an effective method to improve the generalization of machine learning models, especially in the settings where robustness (e.g., generalization across distributions) is valued. Existing literature discussing this 'hard-to-learn' concept are mainly expanded either along the dimension of the samples or the dimension of the features. In this paper, we aim to introduce a simple view merging these two dimensions, leading to a new, simple yet effective, heuristic to train machine learning models by emphasizing the worst-cases on both the sample and the feature dimensions. We name our method W2D following the concept of 'Worst-case along Two Dimensions'. We validate the idea and demonstrate its empirical strength over standard benchmarks.
AB - Training with an emphasis on 'hard-to-learn' components of the data has been proven as an effective method to improve the generalization of machine learning models, especially in the settings where robustness (e.g., generalization across distributions) is valued. Existing literature discussing this 'hard-to-learn' concept are mainly expanded either along the dimension of the samples or the dimension of the features. In this paper, we aim to introduce a simple view merging these two dimensions, leading to a new, simple yet effective, heuristic to train machine learning models by emphasizing the worst-cases on both the sample and the feature dimensions. We name our method W2D following the concept of 'Worst-case along Two Dimensions'. We validate the idea and demonstrate its empirical strength over standard benchmarks.
KW - Self-& semi-& meta- Representation learning
UR - http://www.scopus.com/inward/record.url?scp=85141750306&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85141750306&partnerID=8YFLogxK
U2 - 10.1109/CVPR52688.2022.00941
DO - 10.1109/CVPR52688.2022.00941
M3 - Conference contribution
AN - SCOPUS:85141750306
T3 - Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition
SP - 9621
EP - 9631
BT - Proceedings - 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2022
PB - IEEE Computer Society
T2 - 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2022
Y2 - 19 June 2022 through 24 June 2022
ER -