Williams and Jonathan Holton recorded their second straight double-doubles.
Williams had 19 points and 10 rebounds. Carter also scored 19 points. Holton added 12 points and 12 rebounds. Daxter Miles Jr. had 11.
WVU (2-0) never trailed but the Dukes hung around for most of the game.
Shakir Brown dropped a 3 and Winston Grays III hit a driving layup 19 seconds apart to bring James Madison within 66-58 with nine minutes to go.
The Mountaineers went on a 6-0 run to take a 76-61 lead with 4:38 to play. Miles made a layup off a pass from Esa Ahmad. Holton hit two free throws and then Williams dunked off a pass from Holton.
WVU jumped out to an early lead with Carter’s alley oop dunk. Back-to-back baskets by Holton and Williams helped the Mountaineers take a 10-3 lead.
Carter made a fast break layup to put WVU up 12 with less than six minutes to play in the first half.
JMU refused to go away and would pull within 10 at the half, trailing 48-38.
Brown came off the bench to score 21 points for JMU. Grays tallied 12 points and Ron Curry added 11.
The Mountaineers improved to 8-3 in the series against the Dukes that started in 1982.
WVU will take on Stetson on Friday at 7 p.m. at the Coliseum.