Windows often has issues removing folder where nested paths are longer than 256 characters. Two quick ways to remove the folder:

cmd /C ("rmdir /S /Q " + $problematicDir)

mkdir empty
robocopy .\\empty .\\weblogic /MIR
rm .\\empty; rm .\\weblogic