This is not a scenario I’ve encountered … not too many of us will ever have more than 256 virtual NICs on a single host. Microsoft has posted a support article on this scenario:
Windows Hyper-V server has a default limit of 256 dynamic MAC addresses. You have a Windows Server 2012 (WS2012) host which is configured with the Hyper-V role. The Hyper-V server is configured to provide dynamic MAC addresses to the guest machines:
When you start a guest virtual machine, you may encounter the following error message:
The application encountered an error while attempting to change the state of ‘<Virtual machine name>’
Synthetic Ethernet Port (Instance ID CCE417C5-BDD9-4216-85CA-248620EE75C6): Failed to power on with Error ‘Attempt to access invalid address’.
On a Windows Server 2008 or Windows Server 2008 R2 Hyper-V host, an Event ID 12565 from source “Microsoft-Windows-Hyper-V-Worker” is logged.
Here’s how this issue is caused. Each host has a default dynamic pool of MAC addresses. This pool is generated as follows:
- The first three octets of the MAC address pool are 00:15:5D. This is unique to all Microsoft “devices”, complying with IEEE standards.
- The next two octets (4 and 5) in the MAC address pool are derived from the IP address of the host (the last 2 octets of the IP address to be precise). This gives the pool some uniqueness in your server farm. We now have 5 of the 6 required octets for a MAC address.
- Finally, the last octet in the MAC address pool is the range 0x0-0xFF. Each instance of this range is used once per virtual NIC (assuming that it’s using a dynamic MAC) on this host
Octets 1 to 3 are unique to Microsoft “devices”. Octets 4 and 5 are for giving the MAC address pool uniqueness for the pool range. And octet number 6 is used to make each dynamic MAC address unique on the network.
If you want to break out a scientific calculator or if you know your computer science, the clue to the cause is in that last piece of the puzzle. We only have octet 6 for each dynamic MAC address instance that can be allocated. An octet is 8 bits, from 00000000 to 11111111. That is 00 to FF in hex. Which is 0 to 255, or 256 numbers. And that means each host can have 256 MAC addresses, by default.
There are workarounds to this, if you are in the very rare situation where you need more than 256 MAC addresses on a single host.
Use Static MAC Addresses
Turn off (shut down) the VM and assign static MAC addresses. System Center VMM makes this easy with centrally human managed pool of MAC addresses, something like with DHCP.
Manually Modify The Per-Host Dynamic MAC Address Pool
Be very careful with this! You don’t want to create overlapping pools of MAC addresses to confuse ARP on your network. The MAC address range is defined by two registry values in HKLMSoftwareMicrosoftWindows NTCurrentVersionVirtualization:
- MinimumMacAddress
- MaximumMacAddress
In my example (above), you can see the range runs from 00:15:5d:01:86:ff. 00:15:5d is the manufacturer unique 3 octets for Microsoft “devices”. 01:86 is unique to the range on this host. And 00 to ff is the default range that limits us to 256 MAC addresses.
An interesting thought that came to me after posting this: you might want to be careful if using DHCP for your hosts – seriously, you should not do this! I can imagine how two hosts could end up accidentally with the same default range if HostA has Address1 when it is initialised, and then Address1 is allocated to HostB when it is initialised.
Microsoft says that you can modify this range … but as I warned before: be careful not to overlap over ranges or devices on your network! You can do this by modifying the fifth and or sixth octet of the default calculated dynamic MAC address range. Changing the fifth octet is the risky one … remember that the 4th and 5th octets are unique per host.
—
If you don’t have this massive environment then don’t touch these dynamic MAC address ranges unless you have to. If you do have to, then (in my opinion) you should be using System Center. A SQL whiz might be able to run a query in ConfigMgr to report on existing MAC addresses from physical devices. PowerShell will come in handy if you want to get the details exiting MAC ranges on your Hyper-V hosts. Stack Overflow has a script example to query lots of servers that you could tweak. 4sysops has another script example. I haven’t tested them, but you can play to your heart’s content 
For VMM installs: Don’t forget to checkbox the “static MAC” othwise you still have the same situation of Hyper-V managing your MAC address pool.
And for the 256 MAC adresses limit: We use clusters of upto 60-70 VMs, each VM has 2 NICs, each VM might have 2 iSCSI NICS: 2x2x70=280 🙂
You can also see and modify the MAC Address range in the Virtual Switch Manager inside Hyper-V Manager.
look at this: http://itproctology.blogspot.com/2011/12/dealing-with-hyper-v-default-mac-pool.html
It puzzles me that It’s fairly difficult to find any mentions of this “potential issue” from Microsoft, when using DHCP to install the server – which lets face it – most of us do. In our case, we use a 1gbit onboard NIC to PXE boot and install Windows on all our hyper-v hosts. If the Hyper-V role is applied BEFORE the 10G Nics are configured (with a static management IP) this could, as Aidan points out, be a potential disaster for your virtual environment. MAC collisions are fairly annoying to troubleshoot, and it’s not always apparent that what you are experiencing IS a result of MAC conflicts. So people, BE WARE! 🙂