Re: Test case draft for Basic networking test case

On 09-12-2025 00:19, Brandon Nielsen via test wrote:
> On 12/8/25 9:31 AM, Kamil Paral wrote:
>> On Fri, Dec 5, 2025 at 6:25 PM Adam Williamson 
>> <[email protected] <mailto:[email protected]>> wrote:
>>
>>     On Fri, 2025-12-05 at 17:14 +0100, Lukas Ruzicka wrote:
>>      > Hello,
>>      >
>>      > The QA team has been missing test cases to test Basic networking.
>>     I have
>>      > written a draft of the test case and now I am collecting
>>     suggestions on how
>>      > to improve it, check if anything is missing, or if something
>>     needs to be
>>      > changed.
>>      >
>>      > The Basic networking test case is here:
>>      > https://fedoraproject.org/wiki/QA:Testcase_Basic_networking
>> <https://fedoraproject.org/wiki/QA:Testcase_Basic_networking>
>>      >
>>      > We will also need to create a WIFI test case and a VPN test case,
>>     both the
>>      > CLI and GUI variants, so if anybody has ideas how to do it, let
>>     me know.
>>      >
>>      > Thanks for your help.
>>
>>     Thanks for writing this!
>>
>>     It seems...long. :D I get that it's trying to encapsulate 
>> everything in
>>     the criterion, but it's a bit overwhelming. Still, it is very
>>     comprehensive and robust.
>>
>>     I think when I wrote the criterion I was envisaging something a bit
>>     shorter, sloppier and vaguer, that mostly relied on the tester's
>>     'typical' network configuration and was just 'try and open a web 
>> site,
>>     does that work?' kinda stuff. But...maybe this is better?
>>
>>     Curious to know what others think. Do we prefer something very 
>> detailed
>>     and 'clean room' like this?
>>
>>
>> Hmmm. First of all, thanks, Lukas, for clearly giving it a lot of 
>> effort! Unfortunately, I'm afraid that this amount of effort will 
>> make it one of those "nobody wants to do it" test cases, because it's 
>> long and complex. But it depends what we want to do with this. If 
>> this is to be automated, then big +1 from me, let's do it. If this is 
>> intended to be a manual test, then I think we need to simplify it, 
>> and not just because of length, that's just one of my concerns. One 
>> of my other concerns is that this is... too short. Networking 
>> requires a lot of know-how and in many areas it seems it doesn't go 
>> deep enough, which means further expansion to explain some steps in 
>> more detail would be needed. For example, it uses keywords like 
>> "enp1s0" but doesn't explain how to figure out what *your* actual 
>> network device is called. The same goes for IP ranges, connection 
>> names ("Wired connection 1"), and maybe some more.
>>
>> Using VM as a remote server is of course the most straightforward 
>> idea, but it has its own pitfalls. For example, ping doesn't work in 
>> libvirt user session, only system session (at least according to my 
>> past experience). The IP ranges will vary (unlike in the testcase, 
>> mine is 192.168.124.0/24 <http://192.168.124.0/24>). And of course, 
>> the tested system is often a VM itself. Does this testcase assume 
>> nested virt in that case? Or VM<->VM connection? This will definitely 
>> get even more complex, if we want this to be followed by the general 
>> public with heavily varied networking environments.
>>
>> I also see two possible goals of this test case, and I'm not sure 
>> which one was the intended one. The first goal is to verify that the 
>> very networking basics work, setting an IP address, ping, curl, etc. 
>> This test case verifies that. The other goal is to test that the 
>> real-world network works on a particular device of the tester. So 
>> actually sending packets to your router and to the internet, 
>> receiving a web page, etc. Since this test case uses a local VM, are 
>> those packets even going through the network card, or are they just 
>> "virtualized" in the kernel? I have no idea. But it surely doesn't 
>> test that your wifi connection works fine, or cable speed negotiation 
>> with your router, and that you can open fedoraproject.org 
>> <http://fedoraproject.org>. The first goal is great for automation, 
>> the second goal is good for human testers with varied hardware.
>>
>> It might be best if we can do both. Convert this into an OpenQA 
>> script that will run tirelessly each compose. The steps are exact, no 
>> need for a human tester to repeat it. And let's have a simplified 
>> version for humans, that will test connecting to a wifi/cable, 
>> pinging a well-known server, opening a website (note that we already 
>> have https:// fedoraproject.org/wiki/QA:Testcase_desktop_browser 
>> <https:// fedoraproject.org/wiki/QA:Testcase_desktop_browser> for 
>> that, though), and possibly some optional stuff like switching a dns 
>> server. The wifi seems like the most important stuff to me (for human 
>> testing), because we can't easily replicate that in OpenQA (most of 
>> the other stuff we can).
>>
>>
>
> I generally agree with all the above, it looks like "too much" and 
> likely to drive people off. If it could be largely automated, that 
> would be excellent.
>
> That being said, I was able just now to complete the tests between two 
> usermode VMs created and run via Boxes. It was pretty straightforward 
> to complete.
>
> As for the wording of the test case itself, I would like to see the:
>
> `sudo nmcli con add type ethernet ifname enp1s0 con-name static`
>
> invocation added to the "Set up static networking on the tested 
> machine (nmcli)" section as well so you can just copy paste the 
> following commands.
>
> It might also be nice if the "Deactivate the existing connection" 
> verbiage made it clear "existing" isn't some network-manager magic, 
> and that you need to figure out your "existing" connection name via 
> `nmcli connection show` as is made explicit in the server 
> configuration section.
>
detailed testing layout is a must.

maybe broken/arranged into categories based on "layers" of networking

-- 
_______________________________________________
test mailing list -- [email protected]
To unsubscribe send an email to [email protected]
Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/
List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines
List Archives: https://lists.fedoraproject.org/archives/list/[email protected]
Do not reply to spam, report it: https://pagure.io/fedora-infrastructure/new_issue