This phase II trial tests how well pB1-11 and human papillomavirus tumor antigen (TA-HPV) vaccines in combination with pembrolizumab work in treating patients with oropharyngeal cancer that has come back (recurrent) or that has spread from where it first started (primary site) to other places in the body (metastatic) and that is PD-L1 and human papillomavirus (HPV) positive. Oropharyngeal cancer is a type of head and neck cancer involving structures in the back of the throat (the oropharynx), such as the non-bony back roof of the mouth (soft palate), sides and back wall of the throat, tonsils, and back third of the tongue. Scientists have found that some strains or types of a virus called HPV can cause oropharyngeal cancer. pBI-11 is a circular deoxyribonucleic acid (DNA) (plasmid) vaccine that promotes antibody, cytotoxic T cell, and protective immune responses. TA-HPV is an investigational recombinant vaccina virus derived from a strain of the vaccina virus which was widely used for smallpox vaccination. Vaccination with this TA-HPV vaccine may stimulate the immune system to mount a cytotoxic T cell response against tumor cells positive for HPV, resulting in decreased tumor growth. Immunotherapy with monoclonal antibodies, such as pembrolizumab, may help the body's immune system attack the cancer, and may interfere with the ability of tumor cells to grow and spread by inhibiting the PD-1 receptor. These investigational vaccines could cause or enhance an immune response in the body against HPV, during which time the activity of pembrolizumab against oropharyngeal cancer associated with HPV may be strengthened. These drugs in combination may be more effective in increasing the ability of the immune system to fight oropharyngeal cancer than pembrolizumab alone.