Windows often has issues removing folder where nested paths are longer than 256 characters. Two quick ways to remove the folder:

cmd /C ("rmdir /S /Q " + $problematicDir)

mkdir empty
robocopy .\empty .\weblogic /MIR
rm .\empty; rm .\weblogic