This study considers how household composition influences the leisure time of men and women in South Africa, using the South African 2010 Time Use Survey. Studying leisure time is important since the allocation of time outside the market provides insights into market behaviour and physical and mental health. Household composition and leisure consumption are highly gendered, with women typically living in larger households and consuming less leisure than men. Regression analysis shows that leisure time allocations are highly dependent on who lives in the household and Oaxaca–Blinder decomposition analysis finds that gender differences in mean leisure time can be attributed to household members, affecting the leisure time of male and female respondents differently. Overall, the results are consistent with traditional gender roles within the household and highlight the lack of intra-household bargaining power for women, providing evidence of gender inequality. Lower leisure consumption for women may have negative implications for their productivity in terms of paid and unpaid work, and for their well-being.