You'll see no difference in brightness with a 3-9x40, 2-7x28, or a 1-4x20 scope with them all set at 4x magnification except possibly in low light with the 1-4x20 if you're young and your iris can open over 5mm in low light. For most adults it cannot. Assuming that the three sopes all have the same design except for the objective lens they'll all have the same field of view and the same eye relief distance. The exit pupil diameters will be 10mm, 7mm and 5mm respectively so the smaller scopes will have less tolerance to radial or axial eye position. If the scope is properly mounted on the rifle to match your hold that's not a bad thing. Smaller exit pupils reduce the amount of potential parallax error.
It's an interesting fact of optical design that no telescope can make the apparent surface brightness of of an extended object brighter than looking at it naked eye. If the light source is too small (in angular size) for they eye to resolve it then a telescope can make the object appear brighter such as when looking at stars. Once an object is resolved (like looking at a distant street lamp), a larger and more powerful telescope will make the lamp look larger, but will not make it appear to have a greater surface brightness. The image though a telescope will allways be a little dimmer han naked eye because of internal reflections off the optical surfaces, though with modern optical coatings those losses are small.