3750-X to 2960-X strange PortChannel behaviour

We have a problem with stacked 2x WS-C3750V2-24TS-S    's ethernet links bundled to port channel to 2960-X access switches (there are multiple of 2960-X's).

The setup is as follows:

Cat3750-X_1 GE 1/0/3 <--------> Cat2960-X GE 1/0/47

Cat3750-X_2 GE 2/0/3 <--------> Cat2960-X GE 1/0/48

Configuration:

3750-X side:

!

interface Port-channel23

description Trunk to sw-access3

switchport trunk encapsulation dot1q

switchport mode trunk

switchport nonegotiate

end

!

interface GigabitEthernet1/0/3

description Trunk to sw-access3

switchport trunk encapsulation dot1q

switchport mode trunk

switchport nonegotiate

channel-group 23 mode active

end

!

interface GigabitEthernet2/0/3

description Trunk to sw-access3

switchport trunk encapsulation dot1q

switchport mode trunk

switchport nonegotiate

channel-group 23 mode active

end

2960-X side:

!

interface Port-channel1

description Trunk sw-core

switchport mode trunk

switchport nonegotiate

end

!

interface GigabitEthernet1/0/47

description Trunk sw-core

switchport mode trunk

switchport nonegotiate

channel-group 1 mode active

end

!

interface GigabitEthernet1/0/48

description Trunk sw-core

switchport mode trunk

switchport nonegotiate

channel-group 1 mode active

end

Port Channel config to the other 3 access switches is identical.

Everything was working great for the last 5 weeks and now we have these problems:

Logs:

3750-X:

Nov 29 10:30:30.434 CET: %LINK-3-UPDOWN: Interface GigabitEthernet2/0/3, changed state to down

Nov 29 10:30:30.711 CET: %LINK-3-UPDOWN: Interface GigabitEthernet2/0/1, changed state to down

Nov 29 10:30:32.044 CET: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet2/0/2, changed state to down

Nov 29 10:30:33.806 CET: %LINK-3-UPDOWN: Interface GigabitEthernet2/0/2, changed state to down

Nov 29 10:31:18.716 CET: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet1/0/3, changed state to down

Nov 29 10:31:18.733 CET: %LINEPROTO-5-UPDOWN: Line protocol on Interface Port-channel23, changed state to down

Nov 29 10:31:19.748 CET: %LINK-3-UPDOWN: Interface Port-channel23, changed state to down

Nov 29 10:31:20.050 CET: %LINK-3-UPDOWN: Interface GigabitEthernet1/0/3, changed state to down

Nov 29 10:31:22.054 CET: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet1/0/2, changed state to down

Nov 29 10:31:22.071 CET: %LINEPROTO-5-UPDOWN: Line protocol on Interface Port-channel22, changed state to down

Nov 29 10:31:23.086 CET: %LINK-3-UPDOWN: Interface Port-channel22, changed state to down

Nov 29 10:31:23.388 CET: %LINK-3-UPDOWN: Interface GigabitEthernet1/0/2, changed state to down

Nov 29 10:32:25.286 CET: %PM-4-ERR_RECOVER: Attempting to recover from loopback err-disable state on Gi2/0/21 (sw-core-2)

Nov 29 10:32:29.747 CET: %LINK-3-UPDOWN: Interface GigabitEthernet2/0/21, changed state to up

Nov 29 10:32:32.666 CET: %LINK-3-UPDOWN: Interface GigabitEthernet2/0/3, changed state to up

Nov 29 10:32:35.744 CET: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet2/0/21, changed state to up

Nov 29 10:32:35.946 CET: %LINK-3-UPDOWN: Interface GigabitEthernet2/0/2, changed state to up

Nov 29 10:32:38.730 CET: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet2/0/3, changed state to up

Nov 29 10:32:39.653 CET: %LINK-3-UPDOWN: Interface Port-channel23, changed state to up

Nov 29 10:32:40.685 CET: %LINEPROTO-5-UPDOWN: Line protocol on Interface Port-channel23, changed state to up

Nov 29 10:32:42.195 CET: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet2/0/2, changed state to up

Nov 29 10:32:42.874 CET: %LINK-3-UPDOWN: Interface Port-channel22, changed state to up

Nov 29 10:32:43.914 CET: %LINEPROTO-5-UPDOWN: Line protocol on Interface Port-channel22, changed state to up

Nov 29 10:33:22.584 CET: %LINK-3-UPDOWN: Interface GigabitEthernet1/0/3, changed state to up

Nov 29 10:33:25.939 CET: %LINK-3-UPDOWN: Interface GigabitEthernet1/0/2, changed state to up

Nov 29 10:33:29.386 CET: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet1/0/3, changed state to up

Nov 29 10:33:31.702 CET: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet1/0/2, changed state to up

2960-X:

Nov 29 10:56:08.535 CET: %ETHCNTR-3-LOOP_BACK_DETECTED: Loop-back detected on GigabitEthernet1/0/48.

Nov 29 10:56:08.535 CET: %PM-4-ERR_DISABLE: loopback error detected on Gi1/0/48, putting Gi1/0/48 in err-disable state

Nov 29 10:56:09.538 CET: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet1/0/48, changed state to down

Nov 29 10:56:10.548 CET: %LINK-3-UPDOWN: Interface GigabitEthernet1/0/48, changed state to down

Nov 29 10:58:08.551 CET: %PM-4-ERR_RECOVER: Attempting to recover from loopback err-disable state on Gi1/0/48

Nov 29 10:58:13.378 CET: %LINK-3-UPDOWN: Interface GigabitEthernet1/0/48, changed state to up

Nov 29 10:58:20.096 CET: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet1/0/48, changed state to up

Nov 29 11:29:49.744 CET: %ETHCNTR-3-LOOP_BACK_DETECTED: Loop-back detected on GigabitEthernet1/0/48.

Nov 29 11:29:49.747 CET: %PM-4-ERR_DISABLE: loopback error detected on Gi1/0/48, putting Gi1/0/48 in err-disable state

Nov 29 11:29:50.754 CET: %LINEPROTO-5-UPDOWN: Line protocol on Interface GigabitEthernet1/0/48, changed state to down

Nov 29 11:29:51.768 CET: %LINK-3-UPDOWN: Interface GigabitEthernet1/0/48, changed state to down

Nov 29 11:29:59.744 CET: %ETHCNTR-3-LOOP_BACK_DETECTED: Loop-back detected on GigabitEthernet1/0/47.

Nov 29 11:29:59.748 CET: %PM-4-ERR_DISABLE: loopback error detected on Gi1/0/47, putting Gi1/0/47 in err-disable state

No errors on any interfaces...:

GigabitEthernet1/0/48 is up, line protocol is up (connected)

  Hardware is Gigabit Ethernet, address is c025.5c64.c8b0 (bia c025.5c64.c8b0)

  Description: 802.1Q Trunk sw-core

  MTU 1500 bytes, BW 1000000 Kbit/sec, DLY 10 usec,

     reliability 255/255, txload 1/255, rxload 1/255

  Encapsulation ARPA, loopback not set

  Keepalive set (10 sec)

  Full-duplex, 1000Mb/s, media type is 10/100/1000BaseTX

  input flow-control is off, output flow-control is unsupported

  ARP type: ARPA, ARP Timeout 04:00:00

  Last input 00:00:00, output 00:00:02, output hang never

  Last clearing of "show interface" counters never

  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0

  Queueing strategy: fifo

  Output queue: 0/40 (size/max)

  5 minute input rate 241000 bits/sec, 154 packets/sec

  5 minute output rate 146000 bits/sec, 67 packets/sec

     328302508 packets input, 249207555791 bytes, 0 no buffer

     Received 27119257 broadcasts (14200187 multicasts)

     0 runts, 0 giants, 0 throttles

     0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored

     0 watchdog, 14200187 multicast, 0 pause input

     0 input packets with dribble condition detected

     193253722 packets output, 34501328920 bytes, 0 underruns

     0 output errors, 0 collisions, 2 interface resets

     2 unknown protocol drops

     0 babbles, 0 late collision, 0 deferred

     0 lost carrier, 0 no carrier, 0 pause output

     0 output buffer failures, 0 output buffers swapped out

Has anyone encountered similar problems? Any help/ideas appriciated.

The updated logs about the flapping MAC very strongly suggest that a switching loop exists in your network. I wonder why STP has not prevented it from occuring. Is there perhaps some server, perhaps MS Windows, using aggregated links? Sometimes a misconfigured station with multiple NICs can cause switching WS-C3750X-48T-S    loops (if some kind of bridging or link bundling is activated on it).