One of our goals in developing natural human-computer interfaces is to allow the computer to expand its vocabulary by learning new linguistic knowledge. This paper presents a system capable of acquiring new language by learning new words, phrases, sentences, and their semantics from users. The acquisition of the new linguistic knowledge at surface and semantic levels is done using multiple input modalities, including speaking, typing, pointing, touching or image capturing. The language knowledge is stored in a rule grammar and a semantic database. Both can be updated periodically with newly acquired language.